Nimr Hammad, a political adviser to Abbas, said that anyone wanted by Hamas would be allowed to remain in the West Bank for their safety, while the rest will be returned "to their families and their homes".
He would not say how many were included in each group or offer an explanation for the move.
Israel said it was Abbas who had originally asked for the men to be allowed to cross the border.
Detained for questioning
Sami Abu Zuhri, a Hamas spokesman, confirmed that the men who were sent back had been detained for questioning but denied that the move was political.
"Those who are accused of breaking the law will be investigated and if they are found to be guilty will be brought to justice. Those who are proved innocent will be released," he told the AFP news agency.
Israeli troops had allowed the Gazans to cross into Israel on Saturday, stripping them first to make sure none were concealing weapons or wearing explosives.
Soldiers prepared stretchers, and ambulances rushed the badly wounded to nearby hospitals.
Mortar shells landed near the Nahal Oz crossing as the Palestinians wait to leave Gaza.
The violence, which lasted most of Saturday, was sparked in the morning when Hamas policemen tried to arrest what they said were suspects in a bombing that killed five Hamas fighters and a girl on a Gaza beach on July 25.
Hamas has blamed Fatah supporters for the beach bombing, but Fatah denies any involvement.
Over the past week, the two sides have engaged in tit-for-tat spates of arrests.
At about the same time as gun battles raged in Gaza, Hamas accused Fatah of kidnapping Mohammed Ghazal, one of its leaders in the West Bank, and warned Abbas that he would be held responsible.
Ghazal's relatives said he was freed after being held briefly by armed men.
His captors were fighters from al-Aqsa Martyrs Brigades, an armed Palestinian group loosely linked to Fatah.
Al Jazeera's correspondent in Gaza, Nour Odeh, said the Hilles clan was closely linked to Fatah and had members serving in al-Aqsa Martyrs Brigades.
"It is an old and prominent clan in Gaza and known to be very proud," she said.
Fatah and Hamas, the two main Palestinian factions, have been deeply divided since Hamas expelled Abbas's security forces from Gaza in a week of bloody street battles in June 2007.
Lamis Andoni, Al Jazeera's political analyst, called the latest violence "an extremely dangerous development" that could trigger an all-out Fatah-Hamas confrontation that could have a serious fallout in the West Bank.
"If it is not contained, prospects of a civil war are imminent, especially if the Gaza clashes spark tribal retributions between the Hilles clan and clans of Hamas security officers," she said.