Hamas - whose charter calls for the destruction of the Jewish state - rejected any suggestion the deal to end its damaging power struggle with rival Fatah could imply it now accepts Israel's existence.
With Israel and the Palestinians preparing for a possible Israeli push into the Gaza Strip following the tank gunner's abduction, there appeared little chance that the agreement over the document could open a path towards peacemaking soon.
Israeli troops massed at the Gaza border while Palestinian fighters planted land mines and piled up mounds of sand as obstacles.
Abbas, the Palestinian president, had sought to soften Hamas's line in the hope of ending the US-led financial siege aimed at forcing the group to recognise Israel, renounce violence and accept peace accords.
During weeks of wrangling in the power struggle, Abbas tried to get Hamas to accept a document penned by Palestinians in Israeli jails which implicitly recognises Israel.
Hamas accepted it only after amendments it insisted would allow it to stick to its "agenda of resistance" to Israel.
"The document included a clear clause referring to the non-recognition of the legitimacy of the Occupation," said Sami Abu Zuhri, the Hamas spokesman, using the group's term for Israel.
Officials close to the negotiations said Abbas of Fatah, and Prime Minister Ismail Haniya of Hamas, drafted a platform accepting a Palestinian state in the West Bank and Gaza Strip, areas captured by Israel in a 1967 war.
"We said we accept a state (in territory occupied) in 1967 - but we did not say we accept two states"
Such a two-state solution to the Israeli-Palestinian conflict would be in line with Fatah's recognition of Israel.
But Hamas legislator Salah al-Bardaweel said: "We said we accept a state (in territory occupied) in 1967 - but we did not say we accept two states."
A senior aide to Abbas said the agreement clearly meant Hamas accepted Israel. Yasser Abed Rabbo accused Hamas of "playing with words in order to save face".
The agreement appeared likely to mean the cancellation of a July 26 referendum Abbas had scheduled, over Hamas's objections, on the prisoners' document. Under the accord, Hamas, which won elections in January, would agree to form a unity administration with Fatah and other factions.
The European Union, main donor to the Palestinians, praised the agreement as a good first step while Washington said it wanted to see more details. Both said Hamas now has to clearly recognise Israel and renounce violence.
David Makovsky of the Washington Institute for Near East Policy said the deal appeared aimed at getting the Europeans to break ranks with the United States to ease the embargo.
"I don't think it will be successful," he said.
Israel has said the document is a non-starter and ruled out dealing with Hamas until the group met those terms.
"The document unfortunately would appear to be just more double-speak," said Mark Regev, Israeli foreign ministry spokesman.