Mahmoud Abbas, the Palestinian president, addressed them at the Muqataa headquarters in the West Bank city of Ramallah.
He told the crowd that a solution to the current political impasse between Fatah and Hamas was near.
Abbas said: "I tell my people that we have achieved great progress on the way to forming a national unity government that can end the siege and open the horizons for political solutions that will end the occupation forever.
"I expect, God willing, that this government will see the light of day before the end of this month."
A "massive" crowd gathered in front of Arafat's resting place, an Al Jazeera correspondent said.
Building the tomb has not yet finished for technical and financial reasons, our correspondent added.
Palestinians are remembering Arafat's life two years after his death in a Paris hospital, aged 75.
Leading Palestinian figures laid wreaths at Arafat's grave, which Palestinians want moved to east Jerusalem, the hoped-for capital of their future state.
Palestinian newspapers plastered Arafat's photos on their front pages Saturday, with the Al-Quds daily proclaiming "How much we miss you Abu Ammar", Arafat's other name.
Abbas, Haniya and other speakers also touched on suspicions that continue to surround Arafat's death. Theories persist that Arafat was poisoned.
Abbas said an investigation into Arafat's death was continuing.
He said: "I repeat here my call for all those doctors involved in the matter to fully cooperate with [investigators] so that our people and the whole world can know the truth of what happened to Abu Ammar."
Arafat is seen as embodying the continued Palestinian struggle for statehood.
Boycotted by Israel and the United States in the final years of his life, his death refuelled hopes for progress on the Middle East peace process.
But two years on, Palestinians' daily lives have grown more desperate.
In the run up to the anniversary, nearly a hundred Palestinians were killed in a week of Israeli army operations, mainly in the Gaza Strip.
Gaza and the West Bank are subject to an aid boycott by Western governments and trade blockades by Israel.
The power struggle between the rival Fatah and Hamas factions has intensified the problems ordinary Palestinians face.