The Old City is particularly packed with shoppers, tourists and people heading for prayers on Fridays.
One witness told Israel Radio: “I saw the wounded [Palestinian] on the ground wasn’t moving at all. I didn’t see anything in his hands. I saw two security guards continuing to shoot at the man, and the man wasn’t moving at all. He was dead.”
David Chater, Al Jazeera’s correspondent in Jerusalem, said the police and the Shin Bet, Shin Bet, the Israeli counter-intelligence and internal security service, have sealed off the Old City.
He said: “They are now searching house to house, trying to find some sort of identification for this man… trying to find out the background to this attack.”
It was unclear whether it was a random assault on the security guard or a planned attack.
The Israeli government employs security guards to protect Jewish families who live in the Arab area of the city.
The walled Old City contains the Haram al-Sharif, or Temple Mount, conplex revered by Muslims, Jews, and Christians.
Israel occupied east Jerusalem in a war in June 1967 and quickly annexed it, declaring Jerusalem the “eternal and indivisble” capital of the Jewish state.
However, its claims are not recognised by the international community and Palestinians hope to make the eastern half of the city, where 200,000 Palestinians live, the capital of their promised state.
The attacked Israeli security guards were near a religious school of Ateret Cohanim, one of several ultra-nationalist Jewish organisations that have bought land and homes from Palestinians through various means in order to place Jews in the Arab neighbourhoods of east Jerusalem.