Forces partially withdrew from the Tal al-Sultan and Brazil areas on Friday, reported Aljazeera's correspondent Samir Abu Shamala. But Israeli armoured vehicles have occupied other areas of the impoverished camp, he said.
Aid workers and medics were collecting Palestinians killed, still lying in streets, after Israel launched its onslaught on Rafah on Tuesday. Ambulance drivers said they were unable to reach the neighbourhoods during the invasion, said Abu Shamala.
Residents returned to survey the damage, finding their homes badly damaged if not totally destroyed.
Said Zurub, the head of Rafah's municipality, said Brazil looked like it had been "hit by an atomic bomb", adding Israeli forces have completely destroyed infrastructure in the area.
Water and electricity supplies in the Tal al-Sultan area are cut off most of the time and food is fast running out, Zurub told Aljazeera.
"Israeli forces are not looking for tunnels in these areas," said Zurub. "They are completely destroying (the areas)."
Israel claimed it launched the invasion to destroy tunnels allegedly used by Palestinians to smuggle weapons from Egypt. Palestinians deny the accusations.
The latest toll from the raid is 55 and dozens of others have been injured, said Aljazeera. Many of those killed were civilians. During the invasion, Israeli helicopters and tanks unleashed missiles and shells at peaceful protesters demonstrating against the raid.
There have been international outcries for Israel to completely end the invasion. Residents of occupied areas urged the international community to intervene, said correspondent Abu Shamala.
"The whole world is doing nothing but watching," Zurub said.