The checkpoint had divided the Mediterranean island before Turkey removed its military from the area.
It was a popular vantage point to view Nicosia's buffer zone, a no man's land of abandoned shops and homes, an area civilians were not allowed to enter.
In 2003, the Turkish Cypriot authorities eased restrictions on visits.
No immediate comment was available from Turkey, although Rasit Pertev, chief adviser to Mehmet Ali Talat, the Turkish Cypriot leader, said: "This is extremely symbolic... The dynamism created by this move will lead to the opening of the crossing."
It was not clear what prompted the Greek Cypriot action, but both sides on the divided Mediterranean island have been under intense pressure for the past few years to strike a lasting peace deal.
Cyprus has been split into an internationally recognised Greek Cypriot south and a Turkish Cypriot north, recognised only by Turkey, since 1974 when Turkish troops invaded to foil an Athens-backed Greek Cypriot coup seeking to unite the island with Greece.
Last December, Turkish Cypriots dismantled a footbridge on the other side of the wall. The move drew protests from the Cypriot government which had wanted security issues to be considered before the bridge was brought down.
Joining the EU
In 2004, Greek Cypriots voted against a UN reunification plan shortly before joining the European Union.
The UN, which has unsuccessfully tried to reunite the island as a federation, said it hoped the border crossing would open soon.
Nick Birnback, spokesman for the UN peacekeeping mission, said: "Our hope is that this will lead to the opening of the Ledra Street crossing in the immediate future."
Ledra Street cuts across the ceasefire line which has divided Nicosia's Greek and Turkish Cypriot residents since inter-communal violence in the 1960s.
Cyprus remains a stumbling block to Turkey's aspirations to become a member of the EU and a source of tension with neighbouring Greece.
"Hopefully people will change and get more used to living together. Even if it takes another 50 years this is a step"
Yusuf Alp, Turkish Cypriot
On Thursday, crowds gathered on both sides to witness the dismantling of the five-metre high concrete barrier, harbouring renewed hopes of a reunification.
Yusuf Alp, a Turkish Cypriot construction worker who heard the news on television, said: "It's a lovely thing to see the wall come down. We've waited years for this.
"Hopefully people will change and get more used to living together. Even if it takes another 50 years this is a step."
Tina Sophocleous, a member of a peace group, said: "This is a very important day for Cyprus.
"This could lead to the re-unification of Nicosia."