"There was a boat that came in close and crashed," he said. "After the crash, the other boat continued away. It didn't stop."
The ferry involved, the Sea Smooth, has a top speed of almost 45 kph (28 mph) and carries up to 200 passengers. Local news reports said it is operated by the Hong Kong and Kowloon Ferry company on a regularly scheduled service.
Hong Kong fire services deployed seven boats, including one to support diving operations, and more than 200 rescue personnel, the government said. Four rescue boats and a team of divers also were dispatched from the mainland Chinese province of Guangdong nearby, China's official Xinhua News Agency reported.
Several dozen relatives gathered at Hong Kong's morgue to await information on their loved ones.
A man who gave only his surname, Lee, said he and several relatives had spent the night searching for his 52-year-old sister, who had boarded the boat with three co-workers from the utility company.
"My niece called me last evening and said she believed my sister was on the boat so we should do something right away, we should go find them," he said.
They went from hospital to hospital, to the pier and a nearby yacht club. On Tuesday Lee was at the morgue, which he said would be the best place to get information.
Victor Li, deputy head of the company that owns Power Assets, the holding company that in turn owns Hong Kong Electric, said the company would provide emergency payments of 200,000 Hong Kong dollars ($25,793) to the family of each person killed.
Li's father, Li Ka-shing, is Asia's richest man. Power Assets Holdings Ltd. and Hong Kong Electric, one of the city's two electrical utilities, are two of several companies in the elder Li's sprawling business empire.
Li Ka-shing visited a hospital Tuesday and told reporters he felt "very sorry."
"I don't want to say too much. I just know that many people have passed away," he said in comments broadcast on Cable TV Hong Kong.
Social media sites lit up with discussion of the tragedy and condolences for the victims and their families. Cellphone footage of the partly submerged boat was posted to YouTube.
Lamma is the third-biggest island in Hong Kong and near one of the coastal Chinese city's busiest shipping lanes. The island is home to about 6,000 people, including many of the former British colony's expatriate workers.
The tragedy is a test for Hong Kong Chief Executive Leung Chun-ying's new Beijing-installed administration. His July inauguration was greeted by protests, and opposition by students and their parents against the proposed teaching of China-influenced patriotic history forced his government to back off the plan last month.
After the collision, Leung rushed to the pier where rescue work was taking place. He said later Tuesday that he would set up a commission of inquiry to investigate the crash.
Follow Kelvin Chan at twitter.com/chanman
- Co-pilot who reportedly crashed plane into...
- Rep. Trey Gowdy: Hillary Clinton wiped email...
- 'A marvellous work and a wonder': A look back...
- Sen. Harry Reid's retirement recalls his...
- Senate's Harry Reid announces he won't seek...
- Death of baby cut from womb in Colorado...
- The Latest: Hospital says it evaluated...
- Therapists share healing powers of horses at...
- 'A marvellous work and a wonder': A... 62
- 'Maternal feminists' defend motherhood... 53
- Texas Sen. Ted Cruz targets... 44
- Rep. Trey Gowdy: Hillary Clinton wiped... 41
- Senate's Harry Reid announces he won't... 34
- Alone at controls, Germanwings co-pilot... 27
- Sen. Mike Lee not ready to endorse Ted... 18
- Preparing for 2016 campaign, Hillary... 17