About 1,400 Palestinians being held in Israeli jails are reportedly on the list to be considered in exchange for Gilad Shalit, an Israeli reservist who was captured by Palestinian fighters last June.
"Every country has spoken about Shalit ... but who knows the names of 11,000 Palestinian prisoners in Israeli jails, who mentions them, who mentions the suffering of their people?" Haniya asked the protesters on Monday.
"Why such a double-standard policy even when talking about a humanitarian issue?"
Several thousand Palestinians gathered outside the Red Cross office in Gaza City for the rally, many holding photographs of relatives held by Israel.
According to recent statistics from the B'Tselem, an Israeli human rights group, there are 9,250 Palestinians in Israeli jails.
Nedal Sarifiti, who said his son Ali is serving a 25-year sentence in Israel on charges of being a Fatah fighters said during the rally: "I am optimistic that the day of freedom is coming."
Ehud Olmert, the Israeli prime minister, when asked in a recent newspaper interview whether Barghouthi should expect an early release as part of a prisoner exchange, responded: "No."
Israeli has repeatedly ruled out releasing Palestinian prisoners convicted of the murders of Israelis.
Barghouthi was convicted of murder by an Israeli court in 2004 over the killing of four Israelis and a Greek Orthodox monk in attacks by Palestinian fighters. He is serving five life sentences.