"There is a greater possibility than many more died than we expected because it happened when they were sleeping," said Taufik Bulu, head of maritime safety in the port of Pare-Pare.
More than 400 people killed after ferry breaks apart and sinks in Java Sea during violent storm.
At least 42 people killed in fire on ferry carrying 330 passengers.
Passenger ship carrying 70 people disappears off eastern Indonesia after reporting engine failure in stormy seas.
Rustam Pakaya, head of the health ministry's crisis centre, said on Monday that six deaths had been confirmed in the accident by Sunday evening.
The ferry was about 50km off Majene, western Sulawesi, when officials lost contact with it around 2am on Sunday.
The 700-tonne Teratai Prima was carrying about 250 passengers and 17 crew members when it sank as it sailed from Pare-Pare in South Sulawesi to Samarinda in East Kalimantan.
"I was travelling with family, but my wife and four of my kids are still missing ... only I survived," one man told The Associated Press.
Sulawesi and Borneo islands have been bashed by storms, heavy rains and high winds for several days amid the tropical wet season.
Ferry transport is crucial in Indonesia, a massive archipelago of some 17,000 islands and 234 million people.
Sea accidents routinely claim hundreds of lives each year, largely due to poor enforcement of safety regulations and overcrowding.
But officials say the Teratai Prima, with a capacity of 300 passengers, was not overloaded.