Moussaab, whose real name is Samir Saioud, was killed during a clash with an army patrol in the Boumerdes region 50 kms east of Algiers on Thursday.
The government said that he was tracked down via information gathered from ex-members of the group pardoned under a recent amnesty.
Algeria said Samir Moussaab was the co-ordinator and second-in-command of al-Qaeda's north African branch, the country's official news agency, APS, reported.
Bombing claims
The group, which changed its name from the Salafist Group for Preaching and Combat (GSPC) earlier this year, claimed responsibility for twin bombings in the capital Algiers that killed 33 people on April 11.
The GSPC began as an offshoot of another group that was waging an armed revolt against the government to establish an Islamic state.
The rebellion began in 1992 after the then military-backed authorities scrapped a parliamentary election an Islamic political party was set to win. Up to 200,000 people were killed in the ensuing bloodshed.
Residents said the Algiers attacks this month were the country's first suicide bombings and heightened fears the north African oil exporting country was slipping back into violence.
The GSPC said it was behind an attack in December near Algiers that killed an Algerian and wounded nine people including four Britons and an American.
It also claimed responsibility for an attack on March 3 on a bus carrying workers for a Russian gas pipeline construction firm that killed three Algerians and a Russian.
Government forces have stepped up assaults on the group's strongholds in the Kabylie region east of Algiers, hoping to wipe out what remains of it after an amnesty for Islamic fighters expired.
Al-Qaeda in the Islamic Maghreb is led by Abdelmalek Droudkel, whose predecessor Nabil Sahraoui, was killed by the army in 2003.