The men were shot dead in the al-Zaytun suburb of southern Gaza City after Israeli armoured vehicles and bulldozers invaded the area.
Palestinian hospital sources and witnesses put the number of dead at eight, but Israeli army sources said 13 people were killed.
Islamic Jihad said four of its fighters were "martyred" during the invasion.
And hospital officials who examined the bodies said some appeared to have been shot in the head at close range.
But an Israeli army spokesman denied the allegations.
He said troops came under attack while conducting an operation to find cells of Palestinian "militants" responsible for attacks on the nearby Jewish settlement of Netzarim.
"The soldiers saw about five to 10 gunmen approach them," he said. "The force opened fire at them. A majority of them were hit."
He also insisted the troops under attack had not left their armoured vehicle throughout the exchanges of fire.
The Israeli army regularly launches
attacks in the occupied territories
However, the killings brought immediate condemnation from the Palestinian Authority, with veteran leader Yasir Arafat's chief adviser, Nabil Abu Rudaina, saying "the Israeli government must bear the responsibility for this massacre".
And Prime Minister Ahmad Quraya also condemned the killings.
"What has happened in Gaza is one of the crimes which Israel commits on a daily basis," he said.
Quraya was speaking to reporters after a meeting with US envoy John Wolf in the West Bank town of Ram Allah.
The violence cast a shadow over his first meeting with Wolf, the man tasked by US President George Bush with overseeing implementation of the troubled "road map" peace plan.
However, Quraya said he was optimistic the impasse in the peace process could be broken, adding he had asked the Americans to help line up a much-delayed meeting with Israeli Prime Minister Ariel Sharon.
"The fact the United States has taken up its important role again of trying to bring both sides to their senses gives us reason to be optimistic," he said.
It was Wolf's first visit to the region since September. The intervening period has seen no progress in the roadmap which was endorsed by all sides at a ceremony overseen by Bush in June.
John Wolf (R) is charged with
reinvigorating the road map
"We came out here to reiterate President Bush's commitment to his vision that he set out in June 2002... That vision is essential for us and we believe it is essential for the two parties," Wolf told reporters.
The envoy held talks on Tuesday with Foreign Minister Silvan Shalom and was to meet on Thursday with Defence Minister Shaul Mofaz.
The road map envisages the creation of an independent Palestinian state by 2005 alongside a secure Israel. But the phased project obliges both sides to meet a series of commitments before any final status agreement can be discussed.
Top-level talks have been frozen since August, when Israel has continued its settlement activity in the occupied territories and targeted resistance group leaders for assassination.
On the other hand, Israel says Palestinian authorities have failed to crack down on resistance groups who have conducted anti-Israel attacks.