The bullet-riddled body of the cleric, identified as Mufti Shamsuddin, 65, was found on Saturday on a street in the poor Orangi district, said area police officer Rasheed Khan.
Shamsuddin was director of a well-known al-Hanafi Islamic science school.
"This appears to be an incident of a targeted killing," Khan said. No claim of responsibility had been made, he said.
Thousands of people have been killed in tit-for-tat attacks by rival Sunni and Shia Muslim fighters over the past 15 years. More than 100 people have been killed in the past year.
Two Sunni clerics were killed last month when attackers on motorcycles opened fire on their van in another part of Karachi.
Most of Pakistan's predominantly Muslim population of 150 million are Sunni; about 15% are Shia.
In what appeared to be another sectarian attack, a suspected Sunni fighter was shot dead days after he was released from jail in the central Punjab province, police said.
Hafiz Hidayatullah Haideri had been in jail after being charged with involvement in an attack on a Shia mosque six years ago.
He was released on bail a week ago and on Friday had gone for a court appearance when he disappeared. He was found shot dead on a road, police in the town of Muzaffargarh said.