The stranded Palestinians rioted in the temporary camps on Monday and had threatened further measures, such as a hunger strike.
Israel was "surprised" by the pilgrims' return and "displeased" with Egypt's decision, an Israeli official said.
Two people, one carrying a large cloth bag, were the first to pass through the Rafah terminal, greeted by green-vested representatives of Hamas, the Palestinian group that rules Gaza.
The two were followed by a flood of returning pilgrims walking across the border.
"Thank God we made it. Our patience led us to results," Samiha Qeshta, 59, an exhausted-looking pilgrim, said.
Israel had been pressuring Egypt not to let the pilgrims through the Rafah crossing because it said some may be carrying arms or money for Hamas.
Three elderly pilgrims have so far died waiting to return home.
A number of the pilgrims told Al Jazeera earlier on Wednesday that they had received Egyptian instructions to transfer them on buses to the Egyptian side of Rafah border crossing.
Earlier in the day, members of the Palestinian Legislative Council staged a sit-in on the Gaza side of Rafah with relatives of the stranded pilgrims, calling for them to be allowed to return to their homes.
Dr Ahmed Bahar, the acting speaker of the council, urged Hosni Mubarak, the Egyptian president, to take up his Arab, national and Islamic responsibilities and immediately end the crisis of the Palestinian pilgrims stranded in El-Arish.
Egyptian security forces were put on high alert at the border crossing.