The wooden boat laden with construction materials and three times the passengers it was built for, sank just 100 metres from the port in Djibouti, a tiny Red Sea state, witnesses said.
It was sailing for the town of Tadjoura, 35km northeast, when it went down shortly around 1pm local time on a pilgrimage known as Djamaad.
Ali Mohamed, a survivor told Reuters: "It was so quick that people were brought down by the materials which sank with them."
The vessel was thought to have been carrying between 250 to 300 people.
Many were missing and feared dead, said an adviser to Ismail Umar Guelleh, the Djibouti president.
"It was so quick that people were brought down by the materials which sank with them"
Ali Mohamed, survivor
"There were many dead," the adviser, Ismael Tani, said by telephone, adding that 69 was a provisional toll.
Tani said the boat was "probably overloaded".
Aljazeera's correspondent in East Africa reported that the boat sank near the Djibouti harbour on Thursday afternoon.
While search teams have fished out many bodies, salvage operations were expected to be halted at night in the absence of proper equipment.
Djibouti, a Horn of Africa nation bordering Eritrea and Somalia, is a former French territory and hosts the only US military base in sub-Saharan Africa.