"There was a boat that came in close and crashed," he said. "After the crash, the other boat continued away. It didn't stop."
The ferry involved, the Sea Smooth, has a top speed of almost 45 kph (28 mph) and carries up to 200 passengers. Local news reports said it is operated by the Hong Kong and Kowloon Ferry company on a regularly scheduled service.
Hong Kong fire services deployed seven boats, including one to support diving operations, and more than 200 rescue personnel, the government said. Four rescue boats and a team of divers also were dispatched from the mainland Chinese province of Guangdong nearby, China's official Xinhua News Agency reported.
Several dozen relatives gathered at Hong Kong's morgue to await information on their loved ones.
A man who gave only his surname, Lee, said he and several relatives had spent the night searching for his 52-year-old sister, who had boarded the boat with three co-workers from the utility company.
"My niece called me last evening and said she believed my sister was on the boat so we should do something right away, we should go find them," he said.
They went from hospital to hospital, to the pier and a nearby yacht club. On Tuesday Lee was at the morgue, which he said would be the best place to get information.
Victor Li, deputy head of the company that owns Power Assets, the holding company that in turn owns Hong Kong Electric, said the company would provide emergency payments of 200,000 Hong Kong dollars ($25,793) to the family of each person killed.
Li's father, Li Ka-shing, is Asia's richest man. Power Assets Holdings Ltd. and Hong Kong Electric, one of the city's two electrical utilities, are two of several companies in the elder Li's sprawling business empire.
Li Ka-shing visited a hospital Tuesday and told reporters he felt "very sorry."
"I don't want to say too much. I just know that many people have passed away," he said in comments broadcast on Cable TV Hong Kong.
Social media sites lit up with discussion of the tragedy and condolences for the victims and their families. Cellphone footage of the partly submerged boat was posted to YouTube.
Lamma is the third-biggest island in Hong Kong and near one of the coastal Chinese city's busiest shipping lanes. The island is home to about 6,000 people, including many of the former British colony's expatriate workers.
The tragedy is a test for Hong Kong Chief Executive Leung Chun-ying's new Beijing-installed administration. His July inauguration was greeted by protests, and opposition by students and their parents against the proposed teaching of China-influenced patriotic history forced his government to back off the plan last month.
After the collision, Leung rushed to the pier where rescue work was taking place. He said later Tuesday that he would set up a commission of inquiry to investigate the crash.
Follow Kelvin Chan at twitter.com/chanman
- 5 faith facts about Ben Carson: retired...
- Here's a quiz to test your Star Wars...
- Humanitarian Bishnu Adhikari hopeful for...
- The Latest on prophet cartoon case: 1 man's...
- After finding a vehicle they believe to be...
- Clinton: Nation needs to fix broken...
- The Muppets to team up with Lindsey Stirling...
- Britain's new princess can't expect...
- Rage to relief in Baltimore as 6... 40
- 5 faith facts about Ben Carson: retired... 24
- Former tech executive Carly Fiorina... 19
- More Americans spending at least half... 16
- Clinton: Nation needs to fix broken... 15
- Governor, legislators leave 'baggage'... 14
- Clinton agrees to testify on Benghazi,... 13
- Famed neurosurgeon Ben Carson announces... 12