The ship was transporting cattle from Uruguay to the Syrian port of Tartous when it overturned on Thursday.
Rescue teams from the Lebanese navy and the United Nations peacekeeping force in the country (Unifil) rushed to the scene after the accident.
One of the survivors, a Filipino national, told rescuers that the British captain of the vessel had been killed.
"He told us that the ship's engine went down and the captain sounded the alarm and told everyone to jump in the water," a rescue official recounted.
"He said that 10 minutes after they jumped, the ship overturned sideways in very high waves and sank with the captain still on board."
Al Jazeera's Rula Amin, reporting from Beirut, the Lebanese capital, said: "The problem is the weather. Severe thunderstorms are impeding the Lebanese army's efforts to try to save as many people as possible.
"It's dark, pouring rain, and there are very strong winds. Sea waves are as high as three metres.
"[The rescue crews] are surrounded by thousands of dead cattle, and that makes it even harder for the Lebanese army and for Unifil forces that are trying to help."