Indonesia's Elshinta radio station broadcast the sound of relatives weeping as they waited in Samarinda for news of those who had been aboard the ferry.
Jusman Syafi'i Djamal, Indonesia's transport minister, said the captain - who was among those pulled alive from the sea - reported that 150 people jumped off the boat before it sank, but their fate was unknown.
Djamal added that there had been a tropical cyclone that caused tides of five or six metres.
The 700-tonne left Pare-pare on the island of Sulawesi and was heading for Samarinda, on the Indonesia half of Borneo, but got into difficulties and sank 50km off western Sulawesi at about 04:00 local time on Sunday morning (20:00 GMT).
Fishermen rescued 18 passengers and crew who were found drifting in three life rafts, according to Taufik Bulu, the head of maritime safety at Pare-pare port.
Ferries are the principle form of transportation in Indonesia, an archipelago of more than 17,000 islands with a population of 235 million people.
The transport ministry's spokesman said the ferry was not overcrowded because it had room to carry up to 300 passengers.
Accidents are common among ferries in Indonesia where overcrowding and poor safety standards are rife.
Indonesia has come under pressure in recent years to improve its transport sector following several serious accidents.
In October 2007 at least 30 people died when the Acita III capsized off Baubau, another 50 died in a fire on board the Levina I in February 2007 and in December 2006 around 400 people drowned when another ferry, the Senopati Nusantara, sank in the Java Sea.