"Jews, Muslims and Christians alike call this city their spiritual home... There should be no place within these walls for narrowness, discrimination, violence and injustice," Benedict said.
"Believers in a God of mercy ... must be the first to promote this culture of reconciliation and peace, however, painstakingly slow the process may be, and however, burdensome the weight of past memories."
The mass came as Benedict continued his week-long pilgrimage to the Middle East.
Earlier on Tuesday, the pope visited the Western Wall, a solemn place of prayer and contemplation for Jews.
Al Jazeera's Jacky Rowland, reporting from Jerusalem, said the pope placed a prayer between the stones of the wall, following the Jewish tradition.
"It contained an appeal for peace in the Holy Land, the Middle East and for people across the world," Rowland said.
Benedict also met Israel's two chief rabbis, expressing the Catholic Church's commitment to reconciliation with Jews.
"...The Church continues to value the spiritual patrimony common to Christians and Jews and desires an ever deeper mutual understanding and respect through biblical and ideological studies as well as fraternal dialogues," he said.
His comments followed a controversial move to welcome back into the Roman Catholic church a bishop who denied the extent of the Holocaust.
Benedict has also been criticised over the Vatican's move to beatify Pope Pius XII - pope during the time of the Holocaust - whom many Jews blame for not speaking out against the Nazis during the second world war.
|Benedict prayed at the Western Wall [AFP]
The pope has also disappointed many Muslims in the occupied Palestinian territories.
Mohammed Hussein, Jerusalem's grand mufti, called on the pope to work to end Israeli "aggression" after Benedict visited the Dome of the Rock, the third holiest site in Islam, on Tuesday.
"We look forward for your holiness's effective role in putting an end to the ongoing aggression against our people, our land, and our holy sites in Jerusalem, Gaza and the West Bank," Hussein said.
Benedict was the first pope to enter the building, which is usually closed to non-Muslims.
Jews consider the plaza on which the gold-domed mosque stands to be their holiest site, the location of the Second Temple destroyed by the Romans in 70 AD.