At least 12,000 supporters cheered for him on Monday outside the courthouse he was being tried in.
The independent K-FM radio station reported that across the country thousands more opposition supporters celebrated in the streets of key towns in eastern, northeastern and southwestern Uganda.
Besigye - the first credible challenger to Yoweri Museveni 19-year rule - is being tried in the High Court for alleged treason and rape. Separately he is charged by a military tribunal with terrorism and illegal possession of firearms.
He has denied all the charges, which supporters say were trumped up to keep him from running against Museveni, the Ugandan president, in February elections.
John Bosco Katutsi, a High Court judge, on Monday ordered that Besigye be released on bail, ruling that the military court's order to detain him was illegal because it was issued after the civilian court suspended the military tribunal's trial.
Yoweri Museveni has ruled
Uganda for 19 years
After the ruling, Besigye raised his hand and flashed a victory sign in court. Minor scuffles broke out when prison warders tried to keep him in custody, but Besigye managed to leave the chamber to sign release documents.
He returned briefly for a hearing on the rape charge, but it was suspended until Wednesday because of the tension over his release.
Outside the courthouse, police fired bullets and tear gas to scatter a huge crowd of opposition supporters, who broke through a police cordon to greet Besigye as he left the court and waved from the open roof of a sports utility vehicle.
An Associated Press reporter saw military police beating some civilians in an effort to disperse the crowd. The army deployed armoured personnel carriers in the city. Besigye said he was concerned he would be re-arrested.
Besigye was detained in November, shortly after returning from self-imposed exile to campaign for the presidential election.
He faces a maximum death penalty if convicted of either the terrorism or treason charges.
"We shall continue to struggle against oppression. This government is falling"
Ugandan opposition leader
"We shall continue to struggle against oppression. This government is falling.
"There are so many people in illegal detention like me, and we shall struggle by all possible means to restore good governance in this country," Besigye said.
The civilian High Court had suspended the military proceedings after Besigye's lawyers argued he could not receive a fair trial before the military court, which is controlled by trusted aides to Museveni.