Egypt opened the crossing for two days to allow in Palestinians who need medical treatment not available in Gaza, and for Palestinians to return home.
Hundreds of Palestinians had gathered on the Gaza side of the border crossing, hoping to leave the impoverished and heavily-sanctioned strip.
Al Jazeera's Ayman Moyheldin, reporting from the crossing, said: "They have been waiting all day [and] there has been growing frustration that the Egyptian authorities have not been letting in enough Palestinians."
Witnesses said the crowd burst past Hamas security forces at the border but was unable to get through the Egyptian side.
Hamas later restored order on the Palestinian side, but Egyptian authorities had already closed the terminal gate, which was supposed to remain open until Thursday.
The Rafah crossing is the main gateway for Gaza's 1.4 million people to travel abroad.
It has largely remained sealed as part of an Israeli-led blockade of the strip imposed after Hamas seized total control of Gaza from security forces loyal to Mahmoud Abbas, the Palestinian president, more than a year ago.
The closure prevents Palestinians from travelling for medical care, studies and family visits.
In January, Hamas blew up large parts of the border wall between Egypt and Gaza, allowing thousands of people to move in and out of the Sinai region for nearly two weeks before it was resealed.
The Gaza Strip, meanwhile, was set to receive its first cement shipment in a year as Israel slightly eased its blockade on Wednesday under an Egyptian-brokered ceasefire.
|Israel allowed basic goods, like cement, into Gaza on Wednesday [AFP]
Peter Lerner, an Israeli military spokesman, said: "The Karni, Nahal Oz and Erez terminals were reopened this morning."
As part of the truce, Hamas is supposed to stop cross-border rocket fire by Palestinian groups in Gaza.
In exchange, Israel will allow fuel, medicine and other essential goods into the territory, and halt military raids.
Since the six-month ceasefire went into effect on June 19, goods crossings into Gaza have been sporadically or partially opened, but none have been returned to the level of activity they saw over a year ago.
Most recently, Israel halted the transit of goods into Gaza after two Qassam rockets were fired from the north of the strip into an open area in the western Negev on Monday, according to Israel's Ynetnews.
Gaza's three main crossing are each used for specific goods tranfers: Nahal Oz for fuel and gas, Sufa for products like fruits, vegetables and medicine - and Karni for raw construction materials such as as cement.
Limited goods came through Sufa and Nahal Oz last Sunday, nothing has come through Karni for a week.
The Erez passenger crossing - used mainly by diplomats and journalists - has continued to open regularly, but with the exception of a few special permit holders, it is inaccessible to ordinary Palestinians.