The election commission said polling stations opened at 7am (0200 GMT) and will stay open for 12 hours.
The 194,000 eligible voters are being asked whether to switch to a British-style parliamentary democracy or to elect a US-model presidency.
Mohamed Nasheed, leader of the Maldivian Democratic party, said: "It will be a turning point in the history of the Maldives."
"Whatever the people of the Indian Ocean country choose, it will be turning toward a people's government."
Nasheed says that the only way to inject democracy and good governance is to have a prime minister answerable to parliament.
The opposition is wary of giving too much power to another leader, or to Gayoom for another five-year term, and is therefore in support of a British-style parliament.
Gayoom is pushing for a US-style political system, with a powerful executive presidency.
The vote is the culmination of a reform initiative started by the president in 2004 amid street protests in the capital, Male, that threatened to bring down the government.
His other reform efforts over the past three years - legalising opposition parties and allowing them to print newspapers - have been followed by police crackdowns on public rallies and the arrests of dissidents.
Ahmed Shaheed, the foreign minister, said: "To my judgment, this is a very sincere effort by the president to bring the Maldives into the 21st century."
Western diplomats say that they believe Gayoom, who has won six elections but has never faced an opponent, did intend to move towards a multiparty democracy.
Robert Blake, the US ambassador, said: "By and large, things are going in a positive direction."
The Maldives is the wealthiest country in South Asia, with tourism accounting for one-third of the economy.
Half of the population is under 18-years old and reasonably well-educated, but unemployment remains high.
The country's highest point is only 2.4m above sea level and rising ocean levels threaten to inundate some islands.
The islands were devastated by the tsunami of 2004 and were battered again earlier this year by storms.