Monday's blast came when dozens of worshippers were offering Maghrib (sunset) prayers in the mosque located in the same area where a prominent Sunni Muslim scholar was assassinated only one day before.
Witnesses said the blast rocked the Imam Bargah Ali Raza mosque.
Karachi Police Chief Asad Ashraf Malik said a body retrieved from the scene was being examined to determine whether it was that of a bomber.
An investigator at the scene also said there did not appear to be a crater, which suggests a human bomb.
The blast razed a two-room office of a trust that runs the mosque and collapsed one wall in an adjacent area where people wash before praying.
Most of the casualties were taken to three hospitals in private vehicles, while other bodies were retrieved from the debris and ferried off in ambulances.
Shia Muslim leader, Hasan Turabi, said there was no security at the Imam Bargah Ali Raza mosque on Monday despite an earlier blast in Karachi on 7 May - also at a Shia mosque - that killed 20 people.
Information Minister Shaikh Rashid Ahmad said President General Pervez Musharraf had expressed grief over the killings and would take an "important step" in response to the incident.
The bombing triggered riots in at least four Shia-dominated districts of the city.
Hundreds of youths set fire to two police vehicles and a gas station outside the mosque, and then ransacked two government offices, stealing furniture and torching it at a traffic intersection.
Police fired tear gas trying to disperse the mob and six or seven gunshots were heard, but it wasn not clear who fired them.
Interior Ministry spokesman Abd Al-Rauf Chaudhry said it was too early to say who was behind the bombing, but it added that it "could be a reaction to yesterday's incident".
Shia's form a fifth of Pakistan's population of more than 150 million people. In recent years relations with the majority Sunni community have been marred by bloody sectarian clashes.
The targeted mosque is situated only a kilometre from where Sunni scholar Mufti Nizam al-Din Shamzai was killed in an ambush by unidentified assailants who sprayed his private vehicle with bullets, also injuring his son and the driver.
The assassination triggered violent protests and incidents of arson in the Binouri Town area, where authorities had deployed heavy contingents of police and paramilitary rangers to contain the unrest.