Speaking at a joint press conference with Mahmoud Abbas, the Palestinian president, the UN secretary-general said it was essential to open the crossings in order to "sustain life" in the strip.
"Beyond preserving life, we have to sustain life. The closure of Gaza must be lifted [and] the crossing points must be opened, not just to allow goods but to allow Palestinian exports out as well," he said.
Israel has closed cargo and pedestrian crossing points in the coastal territory for long stretches this year after security alerts and clashes with Palestinian fighters, causing widespread hardship for the Palestinian people.
More than 200 Palestinians have also died in fighting with Israel since June, after Palestinian fighters killed two Israeli soldiers and captured a third, Corporal Gilad Shalit, in a cross-border raid.
Annan also lent support to the formation of a Palestinian unity government, which would help ease international sanctions imposed on the territories after Hamas won elections in January.
"If the Palestinians can unite around a common, realistic programme and if it can help bring the security situation under control, it would be a very positive development indeed and the UN will do whatever it can to support you," he said.
Annan said he would discuss the idea with Ismail Haniyeh, the Palestinian prime minister.
No international force
Violence continued on Wednesday in Gaza, with eight Palestinians killed in Israeli air strikes and clashes in Gaza City and at least five people wounded, Palestinian medical sources said.
International officials have proposed deploying a force in Gaza similar to that in Lebanon to stop the violence, however Annan said the UN did not have any "immediate plans for the deployment of troops to the Palestinian territories", instead urging the Palestinians to stop fighting among themselves.
Annan also said he agreed with Abbas that an end to the Israeli occupation of lands the Palestinians claim for a future state and the creation of that state was "the key to resolving the problems of this troubled region".