"Mousavi we support you! We will die but retrieve our votes!" shouted supporters, many wearing the trademark green colour of Mousavi's election campaign.
Later reports said some shots had been fired at the rally.
"There has been sporadic shooting out there ... I can see people running here," a reporter of Iran's English-language Press TV said in a live call from Tehran's Azadi Square.
"A number of people who are armed, I don't know exactly who they are, but they have started to fire on people causing havoc in Azadi Square," he said.
Sadegh Zibakalam, who was present at the rally, suggested the number of people there could even be over one million.
"To be perfectly honest with you I couldn't believe my eyes ... The numbers were simply countless," he told Al Jazeera.
"The atmosphere was very peaceful. Every now and then people wanted to chant slogans in favour of Mousavi or against Ahmadinejad ... but for most of the time the crowd was very quiet."
He said the rally was peaceful.
"Many people had even taken their children. They had come with their whole family," he said, adding that there was word among the crowd that a similar demonstration would take place on Tuesday.
There have been several clashes between Mousavi supporters and riot police since Ahmadinejad was declared the landslide winner, with some indications that a number of violent incidents have taken place around the country.
The official election results gave Ahmadinejad 63 per cent of the vote and Mousavi just 34 per cent, figures which Mousavi has dismissed as a "dangerous charade".
Ahmadinejad delayed attending a regional security summit in Russia as a result of the protests.
Iran has faced a growing international backlash over the validity of the polls.
The European Union has said it plans to demand clarification of Ahmadinejad's victory and both France and Germany have summoned the respective Iranian ambassadors to account for events.
Ban Ki-moon, the UN secretary-general, said that he was closely following the situation to see "how it evolves".
"The position of me and the United Nations is that the genuine will of the Iranian people should be fully respected," he said.
Washington, which is locked in a dispute with Iran over its nuclear programme, is also monitoring the situation.
"Obviously we continue to have concerns about what we're seeing," Robert Gibbs, the White House spokesman, said.
Pro-Ahmadinejad supporters gathered outside both the British and French embassies in Tehran to protest against interference in Iranian affairs by foreign powers.
Some chanted slogans against the "plots of Iran's Western enemies", a witness said.
Mohammad Khatami, Iran's reformist former president, criticised the authorities for denying permission for the pro-Mousavi rally to be held and said the election had dented the trust of the public.
Ayatollah Ali Khamenei, Iran's supreme leader, has ordered officials to look into the election complaints.
The 12-man Guardian Council said it would rule within 10 days on the two official complaints it had received from Mousavi and Mohsen Rezaie, another losing candidate.
The council headed by Ayatollah Ahmad Jannati, who endorsed Ahmadinejad before the vote, vets election candidates and must formally approve the results for the outcome to stand.
Earlier in the day, about 400 pro-reform students, many wearing green face masks to conceal their identity, gathered at a mosque in Tehran University and demanded Ahmadinejad's resignation.
|Pro-Ahmadinejad supporters protested against what they said was foreign interference [AFP]
Some said a religious armed group had attacked their dormitory.
"They hit our friends and took away at least 100 students. We have no news about their whereabouts," said another student.
University officials denied the reported incident, but Iran's ISNA news agency later reported that MPs had called for an investigation into the attack.
Ebrahim Yazdi, the leader of the banned opposition Freedom Movement, said Ahmadinejad's attacks on his opponents had opened a "Pandora's box" which had led to a deep crisis within the Islamic state's establishment.
"The result of such a crisis now is that the rift among the ... personalities in the revolution is getting deeper," he said.
"It is also between people and their government ... a rift between state and the nation. It is the biggest crisis since the  revolution."