"This government has blood on its hands," read a sign carried by some of the protesters on Wednesday in front of the Israeli defence ministry following the attack in Gaza's northern town of Beit Hanoun.
The demonstrators called for "dialogue in place of war", while some called for the resignation of Amir Peretz, the defence minister and a former Peace Now activist who has presided over a bloody four-month operation against fighters in Gaza that has left more than 300 Palestinians dead.
Meanwhile B'Tselem, a prominent rights group in Israel, called for a criminal investigation, saying Wednesday's attack in which several women and children died raised "grave concern that the action constitutes a war crime".
"Artillery fire, which is inherently inaccurate, near a densely populated residential area makes civilian casualties very likely," it said in a statement.
"Therefore, the military's contention that they did not mean to kill civilians in Beit Hanoun is disingenuous lip service."
The Israeli army said it fired artillery shells on an area used by fighters to launch rockets into Israel, with the shells apparently missing their target.
Peretz has ordered an immediate investigation.