The Palestinian Interior Ministry said the protests held up traffic on Gaza's two main north-south roads for about an hour on Saturday.
The incidents were the latest sign of a growing wave of chaos gripping the coastal strip in the run-up to elections on 25 January.
The protesters, who included relatives and former colleagues of the dead officer, demanded that Mahmoud Abbas, the Palestinian leader, popularly known as Abu Mazen, impose order in Gaza.
They also called for the resignation of Nasser Yousef, the interior minister, who oversees Palestinian security.
A spokesman for the armed men, identifying himself as Abu Wasfi, said: "This is a first step and a message to Abu Mazen ... we ask him to use the force of law also and arrest the killers and execute them."
The armed men opened the roads after about an hour.
Abbas refuses to delay the poll,
fearing even more chaos
Abbas has called for an end to the lawlessness. But his security forces, weakened by internal divisions and fighting with Israel, have been unable to restore order.
The plainclothes officers said they are members of the al-Aqsa Martyrs' Brigades, a resistance group linked to Abbas's Fatah party.
The Brigades was set up shortly after the outbreak of the Palestinian uprising in 2000, and was largely controlled by Yasser Arafat, the late Palestinian president.
Many of the recruits were members of the Fatah and the security forces.
Gaza has experienced an upsurge in violence since Israel completed its withdrawal from the area in September.
"This is a first step and a message to Abu Mazen ... we ask him to use the force of law also and arrest the killers and execute them"
Abu Wasfi, spokesman for the armed men
Al-Aqsa gangs have been responsible for much of the chaos, including abductions of foreigners and shoot-outs in the streets, demanding government jobs and other favours as a payment for fighting Israel.
The lawlessness has threatened to undermine the parliamentary vote and drive voters away from Fatah and towards Hamas.
Some elements in Fatah have urged Abbas to cancel the election until law and order is restored. Abbas has rejected the calls, saying a postponement will only intensify the chaos.