In another Chittagong port, called Cox's Bazar, as many as 10 fishing boats with nearly 100 fishermen onboard capsized. An estimated 80 fishermen made it back to shore.
However, Selim Khan, a local government administrator, said that the fishermen who had returned to shore put the figure at 12 trawlers lost.
The country's meteorological department said on Sunday that a monsoonal depression, which hit the Bay of Bengal on Thursday, had reached Orissa, an eastern coastal state in India.
The bulletin said: "Maritime ports of Chittagong, Cox's Bazar and Mongla have been advised to keep the local warning signal hoisted.
"It is likely to move in a north-north-westerly direction further inland and weaken gradually by giving precipitation."
Officials have also warned of flooding, with low-lying areas of several coastal districts, including Chittagong, Cox's Bazar and their offshore islands, likely to be inundated by water surges of up to 1.2m, driven by high winds.
Hundreds of fishermen in Bangladesh have died in storms and many go missing in the Bay of Bengal every year.