Dr Moaiya Hassanain, of the Palestinian health ministry in Gaza, said 22 people, including six Hamas police officers and an 11-year-old girl, were killed and another 150 were wounded.
Hamas confirmed the death of one of its high-level commanders, Abu-Jibril Shimali.
Moussa was killed when fighting resumed after dawn on Saturday, Ihab Hussein, a Hamas interior ministry spokesman, told the Associated Press news agency.
"Abdel-Latif Moussa was killed in an explosion," he said. "It's not clear whether he was killed from an explosives belt he was wearing or from Hamas gunfire."
The fighting began at the Ibn Taymiya mosque after Moussa's speech on Friday.
Hamas security forces seized control of the building after several hours of heavy clashes.
Some of the fighters holed up inside then managed to escape to Moussa's home.
Scores of people were wounded in the fighting outside that building, several of them critically, medical officials said.
Jund Ansar Allah seeks a Palestinian legal system based purely on the sharia and has accused Hamas of being too liberal.
The group is said to have threatened to burn down internet cafes, and has called for people using Gaza's beaches to dress more modestly.
"We are today proclaiming the creation of an Islamic emirate in the Gaza Strip," Moussa had told worshippers at the mosque on Friday.
Moussa said that if Hamas were to implement sharia, he would immediately instruct his followers to comply with the group's instructions.
|The fighting began on Friday at Rafah's Ibn Taymiya mosque after Moussa's sermon [AFP]
An audience of several hundred men filled the mosque as Moussa spoke, cheering and shouting in response to his address.
A spokesperson for the Hamas government's interior ministry dismissed Moussa's comments and described him as being mentally unstable.
Our correspondent said Jund Ansar Allah subscribes to a "very radical and strict interpretation" that is commonly referred to as Salafism.
"It's really the ideological force behind many of these attacks we've seen in Afghanistan and elsewhere," Mohyeldin said.
"It really divides the Muslim world from the Christian and Western world and believes that there is a direct clash of civilisations.
"This group here in Gaza sees itself as part of that wider conflict.
"They subscribe to that philosophy, calling on their followers to adopt it in their struggle to end both Israel's occupation and the occupation of all Muslim lands at the hands of Western powers."