Azzam al-Ahmed, a Fatah politician, and Mousa Abu Marzouk, the Hamas chief negotiator, signed the deal in Saleh's presence at the Yemeni presidential headquarters.
Jacky Rowland, Al Jazera's correspondent in Jerusalem, said the deal, which agrees only to open talks on a number of issues, appeared to bear little substance.
However, she said: "Anything which can be seen as getting Hamas and Fatah to agree to talk to each other has to be seen as progress in itself when one bears in mind the events of the last year."
The two delegations had earlier accepted the initiative as a blueprint for peace but disagreed on practicalities that would restore control of the Palestinian Authority (PA) over the Gaza Strip.
Fatah mainly argued that Hamas should cede control of the Gaza Strip as a precondition to restore the jurisdiction of the Palestinian Authority.
But Hamas argued that withdrawing forces from authority and security buildings in Gaza should be part of an agreement to restore a national unity government.
Yemen's peace initiative proposes seven points for Palestinian reconciliation
• Gaza must be returned to how it was prior to the Hamas takeover last June
• Agreement to hold early elections
• Resumption of dialogue on the basis of the 2005 Cairo agreement and the Mecca agreement of 2007
• Respecting the Palestinian Law and Basic Law and adherence to it by all parties
• Reconstruction of the Palestinian security institutions
• All Palestinian institutions to be free of any factional discrimination, subject to the law and the executive authorities
To prevent the failure of talks in Sanaa, the Yemeni President reworded the initiative to address those concerns while introducing new mechanisms for reconciliation.
Saleh's revised plan stated that a solution should be a wide-ranging deal, including agreements reached by Palestinian factions in Cairo in 2005, as well as the Saudi-sponsored Mecca accord between Hamas and Fatah that led to the formation of a short-lived Palestinian unity government earlier last year.
"Hamas and Fatah have agreed to accept the Yemeni initiative as a framework for dialogue ... and a return of the Palestinian situation to what it was before the events in Gaza [last June]," the declaration said.
The text, which was read out to journalists by Abu Bakr al-Qurbi, the Yemeni foreign minister, said the dialogue also aimed to "reconfirm the unity of the Palestinian homeland in terms of its land, people and the Palestinian Authority".
The deal calls for the restructuring of Palestinian security institutions, with the forces intended to come under the control of the Palestinian Authority and government rather than Palestinian factions.
A committee comprising a number of Arab countries would follow up the implementation of both the Cairo and Mecca deals.
Sanaa and other Arab states that support an inter-Palestinian dialogue would "put pressure" on Fatah and Hamas to hold direct talks on the basis of the Yemeni blueprint, a source told Al Jazeera.
An unexpected upheaval, unrelated to the initiative itself, came when Abbas sent a delegation of the Palestine Liberation Organization (PLO) to the Sanaa talks, prompting Hamas to initially refuse any negotiations.
The PLO, established in 1964, is internationally recognised as the sole representative of the Palestinian people.
Hamas, founded in 1987 in the Gaza Strip, originated outside the framework of the PLO, and has been seen as a challenge to the long standing organisation.
But in 2005, Hamas and the other Palestinian factions reached an agreement in Cairo to achieve representation in the PLO.
Hamas insisted that the Yemen negotiations be between Fatah and Hamas in order for their differences to be resolved.