Malik Nasir, also known as Amir Abu Ayub, was killed by security forces on Sunday, said a security source in Algiers, who declined to be named.
"This is a big fish because he has been in the terrorist network for more than a decade and we are happy that he is neutralised," the source said.
Nasir, who kept a low profile, was suspected of being a founding member of the Salafist Group for Preaching and Combat, which is on the US list of "foreign terrorist organisations".
The Interior Ministry was not available for comment.
Working on information obtained from rebels captured last week, army units tracked down Nasir to a house in the Zarmuri area, some 30km east of the capital Algiers.
"This is a big fish because he has been in the terrorist network for more than a decade and we are happy that he is neutralised"
Algerian security source
Algeria's civil unrest began when the army cancelled legislative elections which an Islamist party was set to win in 1992.
Secular authorities feared a win could turn the oil-rich country into a purist Islamic state.
Authorities estimate that 150,000 people have since died, although violence has sharply fallen in recent years, bringing back much needed foreign investment.
Nasir's death came as the military intensified a sweep on rebel strongholds in the north of Algeria.