Israel was critisised when it captured Arab East Jerusalem in the 1967 Middle East war and annexed it.
It regards all of Jerusalem as its "eternal and indivisible" capital.
Palestinians want East Jerusalem to be the capital of what they hope will be a future Palestinian state in the occupied West Bank and Gaza Strip.
The future of Jerusalem is one of the core issues both the Israelis and Palestinians agreed to tackle at US hosted peace talks in Annapolis, Maryland, in November 2007.
Olmert's comments appeared to be another move to prepare Israeli public opinion for the possibility of a deal that would loosen Israel's control of all of Jerusalem.
However, Ehud Barak, Israel’s defense minister said on Tuesday that although Israel was opening up its roadblocks and checkpoints in the West Bank to Palestinians, it would not make any sweeping changes to the network.
He said the roadblocks, which are condemned by the Palestinians, are necessary to prevent attacks on Israel.
"There is no chance of waging an effective fight against terror without a real and daily control of the field, and the checkpoints and roadblocks will remain," Barak said.
The US has urged Israel to relax travel restrictions on Palestinians in the West Bank.
In a speech on Monday, Mahmoud Abbas, the Palestinian president, voiced frustration at the slow pace of change.
"Each time they say they will remove [checkpoints] or have already removed some of them -but I can honestly say they did not remove a single checkpoint," he said.
Israel and the Palestinians have said they hope to reach a peace agreement before George Bush, the US president, due to visit Israel and the West Bank next week, leaves office next year.