Shots ripped through the air above the protesters as baton-wielding police on Tuesday arrested up to 50 young men.
Those taking part in the demonstration were calling for greater political reforms during the country's first human rights conference.
The London-based Movement for Islamic Reform called for the rally against the detention of opponents.
The official Saudi Press Agency (SPA) wrote off the protest as "a rally by a number of individuals" which "disrupted traffic" in a busy district of the Saudi capital.
One of those arrested said many of the people detained were released after a few hours.
The rare show of public opposition came a day after the kingdom, an absolute monarchy, announced it would hold its first elections to vote for municipal councils.
The announcement by the cabinet under de facto ruler Crown
Prince Abd Allah followed growing demands for reform to allow
wider political participation, elections and freedom of expression in the conservative state.
The authorities also set up roadblocks to prevent protesters from reaching the building where the human rights conference was being held in central Riyadh.
Some women were seen trying to approach the rally, only to be dispersed by the police.
The opposition group timed the protest to coincide with a human rights conference that opened in Riyadh on Monday.
The conference was organised by the Saudi Red Crescent and attracted representatives from around 15 countries, including the United States.
Witnesses told Reuters that protesters, mostly under the age of 30 and wearing traditional Saudi flowing robes, chanted "God is greatest", while demanding reforms.
Saudi soldier mans a gun placed
on top of a military vehicle
Carrying banners, they also called for the release of political prisoners held in the kingdom.
Reformists generally greeted the government decision to hold
municipal elections as a major step forward, but hoped it would not be a one-off move to appease calls for reform.
Saudi analysts said plans for the polls were a signal the
government was serious about expanding popular participation in the kingdom. The cabinet said preparations for the elections should not take more than a year.
The kingdom has mounted sweeping arrests against alleged militants, particularly following the 12 May Riyadh bombing which left 35 people dead.
Saudi Arabia also faces international condemnation over its own rights record and for implementing its interpretation of Sharia law punishments, including public beheadings and discrimination against women.