But Walid Batrawi, Al Jazeera's correspondent in Ramallah, said the meeting has raised few expectations.
He said for many Palestinians, talk was not enough - they needed to see action.
'No trust'
Analyst George Giacaman said: "There is considerable scepticism on part of the Palestinians after the failure of past meetings.
"Also the continued Israeli incursions, assassinations and confiscation of land.
"There is no basic trust in the Israelis' intentions given that also Israel is the one of the few states that its borders have not yet been defined and the Zionist programme in Palestine has not yet finished."
Mahmoud Abbas and Ehud Olmert were originally scheduled to meet on Tuesday, but the talks were postponed due to technical reasons.
Saeb Erekat, a Palestinian negotiator, said the conference "will launch permanent-status treaty negotiations" based on a joint document that will be presented to the forum.
Erekat said the issue of setting timelines would be discussed later.
"I think this will be left for the conference to decide," he said.
Joint document
Olmert, the Israeli prime minister, hosted Abbas, the Palestinian president, at his Jerusalem residence.
After their meeting, the two leaders brought together their negotiating teams for the first time and instructed them to begin drafting the joint document in the coming weeks.
Yasser Abed Rabbo, an adviser to Abbas, said no details about the joint document would be released while the negotiations were under way.
He said the document would eventually be presented to the decision-making body of the Palestine Liberation Organisation (PLO) as well as Olmert's cabinet.
Condoleezza Rice, the US secretary of state, plans to return to the region next week to assess the status of the talks.
The conference is part of a US-led effort to bolster Abbas and his West Bank-based government and to isolate Hamas, which seized control of the Gaza Strip in June.
Tensions with Hamas
Meanwhile, tensions between Hamas and Abbas's Fatah faction were further stoked when three Hamas Executive Force fighters were injured in a roadside explosion in the Gaza Strip early on Thursday.
An Israeli army spokesman said Israel was not involved.
On Wednesday, Hamas ordered the temporary closure of the Fatah headquarters in the Gaza Strip, citing security reasons.
Hamas said the headquarters was located near where a car exploded on Tuesday night, killing three Fatah members.
A Hamas security official accused the Fatah members of trying to plant a bomb near a Hamas security post when their vehicle exploded.