The secular Popular Front for the Liberation of Palestine (PFLP), the Democratic Front for the Liberation of Palestine (DFLP) and the communist People's Party organised the rally.
Demonstrators made their way through Gaza City calling for and end to the bitter divide and waving PFLP banners.
It was unclear as to what prompted the police to open fire to disperse the protesters.
Jamil Majdalawi, a senior PFLP leader, said that he was concerned that animosity between the Fatah and Hamas governments could become permanent.
"We are with Fatah and Hamas when they are right and we are against them when they are wrong," he said.
Divisions between the two factions existed before Hamas forced Fatah from the Gaza Strip in fierce street battles.
Recent Egyptian meditated-talks in Cairo aimed at forming a unity government have failed to bring the factions together.
Lamis Andoni, Al Jazeera's Middle East analyst, said: "The protests are an indication that people are finding the internal divisions intolerable.
"It has become a blatant struggle for power that has little to do with real political differences.
"Palestinian factions like the PFLP and the DFLP fear that the reconciliation talks in Cairo are not succeeding because they are no longer focused on national unity but rather on the terms for power sharing between Fatah and Hamas."