The OSCE mission, which began in 1992 and currently comprises about 200 people, had tried to promote a peaceful resolution to tensions between Georgia and South Ossetia until the hostility led to an armed conflict in August.
Gottfried Hanne, the deputy head of the mission in Georgia, said on Thursday there was a real need for monitoring to continue, because the situation had "not returned to normal" following the fighting.
Hanne said: "As of today, the mandate of the OSCE mission in Georgia expired, but this does not mean that we just turn the key and leave the country tomorrow."
The closing procedure will take several months and Hanne said that the mission hoped that ongoing consultations between Russia and Georgia would allow the monitors' mandate to be renewed.
Alexander Stubb, the foreign minister of Finland, which held the presidency of the OSCE in 2008, said that the deadlock was regrettable.
However, he held out the prospect of negotiations resuming after January 1 as it would take two to three months to dismantle the 200-person mission.
Georgian forces are in an armed stand-off on the de facto border with South Ossetian and Russian forces, and analysts say a new flare-up of fighting is a real possibility.
|There are about 200 OSCE observers in the area [EPA]
Western states say OSCE monitors patrolling Georgia's conflict zone with South Ossetia can provide early warning of any new flare-up, and investigate allegations of rights abuses against ethnic Georgians.
Britain, one of the most vocal European critics of Russia's actions in Georgia, said it would hold Moscow in particular responsible for security in the conflict zone after it blocked the work of the OSCE monitors.
A small force of unarmed military observers from the OSCE and a larger European Union observer contingent were deployed to Georgia to monitor observance of a ceasefire agreement when fighting between Georgia and Russia ended.
Neither group has been allowed into South Ossetia and as a result Western diplomats say the outside world has been left in the dark about what is happening inside the region.
Human rights groups say South Ossetian militias, with the complicity of Russian forces, have systematically destroyed ethnic Georgian villages and harassed the few ethnic Georgians who remain.
Moscow and separatist authorities deny the accusations.