"Georgia doesn't have a legitimate president," Levan Gachechiladze, the main opposition leader, told supporters.
Saakashvili won the January election with 53.47 per cent of the vote, while Gachechiladze had 25.67 per cent, according to official results released on Sunday.
Gachechiladze and his supporters denounced the official count, saying it reflected a massive effort by the government to rig the vote.
They called for the election officials responsible to be prosecuted and demanded a runoff vote between Saakashvili and Gachechiladze.
"Our fight will continue and we will not allow Saakashvili to steal our votes," Gachechiladze told the crowd.
Helena Bedwell, a journalist in Tiblisi, told Al Jazeera the rally was "much anticipated".
"Tens of thousands attended today, not just from the capital Tiblisi, but also from the regions ... it was joined by other presidential candidates who also lost the race against Saakashvili."
She said the rally ended peacefully, but noted further demonstrations had already been planned.
The opposition has promised another rally on Tuesday outside the headquarters of Georgia's main state-controlled broadcaster in order to demand more air time for their candidates.
Western monitors said the vote was competitive and broadly fair despite some violations, but the opposition disagreed and said it was fixed.
"Saakashvili has rigged the election," Lamara Bliadze, a demonstrators at the rally, said as she listened to the opposition leaders' speeches.
"He has to listen to the opinion of the people."
The Georgian president, an ally of the US who came to power following the peaceful "Rose revolution" in 2003, called the election in November after he ordered police to violently suppress a five day long anti-government protest in Tblisi.
As president, Saakashvili has pursued liberal economic policies, moved the country toward membership of both Nato and the EU and instituted reforms that have attracted sizeable foreign investment.
But many Georgians say they have missed out on the profits generated by such investment and accuse Saakashvili of running a corrupt, elitist government which has handled the economy poorly.
Opposition leaders have been trying to galvanise support since the election by holding a series of protests, but the rallies have had smaller turnouts than hoped for and Sunday's protest attracted far fewer than the 100,000 people they had predicted would attend.