Police said the attack on Wednesday, that also left two people wounded, occurred at the Muhammadi mosque in the neighbourhood of north Karachi.
"It's too early to say whether it's an act of terrorism or the result of personal enmity. We are investigating," police officer Imran Ahmar said.
Sectarian rivalry between Shia and Sunni Muslims has exploded in a wave of attacks in Pakistan in recent months and dozens of people have been killed in the violence.
Some 250 worshippers were said to be in the mosque at the time of the attack.
"Someone started shooting during the prayers," prayer leader Mufti Faruq Ahmad Siddiqui said. "I only heard gunshots on my back ... then the people started running."