"I want a peaceful transition," Nasheed, a former political prisoner and outspoken critic of Gayoom, told reporters as results were still coming in.
"I want my supporters to be calm."
Gayoom later conceded defeat and congratulated Nasheed.
"Beloved citizens of the Maldives,I accept the results of the October 28 run-off election and respectfully congratulate Mohamed Nasheed and his party," Gayoom told the state-run Voice of Maldives radio. He said he would not leave the country.
In the first round of voting three weeks ago Gayoom, the leader of the ruling Dhivehi Rahyithunge party (DRP), took 40 per cent of the votes but failed to secure an outright victory.
Vote for change
Nasheed has promised to root out corruption, deliver better healthcare and communications to remote islands, cut state spending, privatise state trading enterprises and turn the lavish presidential palace into a university.
Many saw the vote as an opportunity to bring about change.
"Gayoom was the president when I was born," Sabra Noordeen, an opposition activist, said. "I'm keen to see a change so I went early and voted [Nasheed]."
Hussein Iqbal, a hotel employee, said he waited an hour to vote "for change".
Nasheed, who was once held for three years and described by Amnesty International as a "prisoner of conscience", had concentrated his campaign in Male, while Gayoom toured the country's outlying atolls in an attempt to secure support from conservative voters.
Ibrahim Shahib, a private sector employee, said he voted for Gayoom, who "may not be great, but still he is the better candidate".
|Gayoom was Aisa's longest-serving president[File: AFP]
Al Jazeera's Stephanie Scawen reported that the incumbent had been counting on support from women and the older electorate, but younger voters said they wanted change and would prefer anyone other than Gayoom in power.
Gayoom, Asia's longest-serving president, has been praised for helping bring major development to the small nation of 1,190 coral islands.
But while the nation has the highest per capita in South Asia, it faces a housing shortage, rising crime and drug abuse.
Gayoom, who is credited with holding the country's first ever multi-party presidential election, had been accused by the opposition of using corruption and strong-arm tactics to maintain his rule.