"It would have been very appropriate for Barak to go. Maybe Barak couldn't go because he is busy planning more [settlement construction] and more incursions," Saeb Erekat, senior Palestinian negotiator, said.
Before the meeting began, Gilad told Israel Radio that it was "part of the general dialogue between us and the Palestinians and us and the Americans" and part of an effort to "make progress in the peace process."
"The stress is on practical talks ... with the aim of moving forward the peace process," Gilad said before the two-hour session.
"All rumours about tensions are baseless."
A statement from Fayyad's office said he was frustrated by Israel's failure to implement a settlement moratorium.
"A freeze on settlement activity is crucial to preserving the possibility of a Palestinian state," the statement quoted him as saying. "By freeze, I mean not one more brick."
Fayyad's statement went on to praise Fraser, saying he showed "seriousness and commitment".
Israel's defence ministry released no official statement after the meeting.
The US consulate in Jerusalem said the sides had discussed "where the parties are not meeting their commitments and the reasons why", and that the meeting included "a cordial but frank exchange of views".
Under the 2003 plan the Israelis were to freeze construction in West Bank settlements and remove some of the more than 100 unauthorised outposts set up by settlers since the 1990s.
The Palestinians were required to dismantle armed groups.
The current talks between Israel and the government of Mahmoud Abbas, the Palestinian president, were launched at a US led summit in Annapolis, Maryland, last year.
This is the first time Fraser has met with both sides and the first time Israeli and Palestinian officials have met since the surge in violence in February.
He was appointed by Condoleezza Rice, the US secretary of state, to oversee the road map plan's implementation.
On the eve of Friday's meeting, US officials said that Washington was not satisfied with the pace at which Israel is moving to implement the road map.
Washington has been critical of Israel's decision to push ahead with settlement expansion on occupied land.
The meeting was the first time Israeli and Palestinian teams had since more than 120 Palestinians, many of them civilians, were killed in a recent Israeli offensive into Gaza.
Israel said the raids were a response to Palestinian rocket fire.
Further Israeli military raids in the West Bank this week killed five Islamic Jihad fighters.
Mahmoud Zahar, a Hamas leader, warned on Friday that continued Israeli attacks on fighters in the West Bank would only fuel violence, rather than reduce it.
"If they don't stop their attack ... they will be eaten up by the fire of resistance," he told a rally in northern Gaza.