Roy Almine, a police officer who helped in the rescue, said huge waves and strong monsoon winds suddenly hit the boat, causing it to overturn and toss passengers into the sea.
"There was no rain. It was like a whirlwind, and the boat overturned,'' Almine said.
Navy, coast guard and local authorities are continuing to search the area between Masbate and Luzon island.
The accident comes four months after the 23,000 tonne inter-island ferry, Princess of the Stars, capsized during a typhoon off the central island of Sibuyan carrying 850 passengers and crew.
Only 57 passengers and crew survived the accident, which was the worst maritime disaster in the Philippines for 20 years.
Mishaps involving ferries are common in this archipelago where many poor people rely on small, poorly-maintained vessels to travel between islands.