His decision to resign is being seen as a goodwill gesture towards Hamas, as the group had repeatedly demanded that Fayyad step down.
"This step comes in the efforts to form a national conciliation government,'' Fayyad said.
However, Hamas officials were dismissive about his resignation.
Fawzi Barhoum, a spokesman for the group, said Fayyad's government was unconstitutional and worked for private interests.
Al Jazeera's Nour Odeh, reporting from Ramallah, said: "The timing may be a little surprising. However, the people certainly expected Salam Fayyad to step down and make way for a government that will enjoy the acceptance of all Palestinian factions.
"The prime minister had already announced that he will put his fate of his cabinet at the hands of president Mahmoud Abbas in preparation for a government of national unity by the end of this month," she said.
Mustafa Barghouti, secretary-general of the Palestinian National Initiative, welcomed Fayyad's decision to stand down.
"If we fail in the national project ... we face disaster"
Ghazi Hamad, Hamas spokesman
"The resignation is a positive, constructive step to support the national dialogue, because the aim of the dialogue within three weeks is to come up with the formation of a national unity government.
"It is natural to say that this government will resign, opening the door for a new unified government."
Delegations from Fatah and Hamas, as well as other Palestinian groups, have set up committees to look at forming a unity government and holding elections.
Ghazi Hamad, a Hamas spokesman, told Al Jazeera: "I think there is a strong determination among Hamas, Fatah and all Palestinian factions to succeed because there is no other alternative.
"If we fail in the national project and dream for the Palestinian people ... then we face disaster and catastrophe."
A planned unity government by Fatah and Hamas will deal with foreign governments, co-ordinate reconstruction in the Gaza Strip and prepare for Palestinian presidential and legislative elections.
"His resignation means that there is enough reason to believe that this dialogue will work, that an agreement will be reached and that Palestinians after almost two years of division will finally have a united government once again," Odeh said.
Analysts say the desire on the part of Palestinians to achieve reconciliation between their divided groups has grown since Israel's three-week military offensive in Gaza, which ended on 18 January.
Fatah and Hamas have long been rivals but their feuding boiled over in June 2007 when Hamas seized control of the Gaza Strip by driving out security personnel loyal to the Palestinian president.