A 14-year-old boy and an unidentified woman were killed instantly by the blast, which went off near the shrine at about 15:30 GMT on Monday, leaving a large crater in the ground.
The blast sparked scenes of panic at the Hazrat Shah-Jalal shrine complex, which was packed with several thousand worshippers for a Muslim saint's festival.
Police said the "quite powerful" bomb was buried in soil near the complex. The shrine had been evacuated and officers were searching for more explosive devices, said a police source who did not want to be named.
No arrests had been made and police did not known who may have planted the bomb or why.
About a dozen people have died in unexplained explosions in the country in recent years.
The large Hazrat Shah-Jalal shrine, in the Sylhet tea-growing district, is a well-known landmark in Bangladesh.