"We came here to oust the soldiers and the soldiers stepped back," Arisman Pongrungrong, a protest leader, said.
"We have made one step towards victory and we'll keep putting on the pressure until parliament is dissolved."
Withdrawal played down
Suthep Thaugsuban, Thailand's deputy prime minister in charge of national security, played down the withdrawal as an "adjustment". He said the troops would return later in the day.
"Right now they have to move out to avoid a confrontation," Suthep said, speaking from the army barracks on Bangkok's northern outskirts where the government is based during the protests.
But buoyed by their success, protesters vowed to take their movement to Abhisit Vejjajiva, the Thai prime minister, by rallying at the barracks on Sunday.
"We hope we will meet with Abhisit," Veera Musikapong, another protest leader, said.
"I hope tomorrow will be the end of this political rally."
The red shirts, who are mainly loyal to Thaksin Shinawatra, the ousted prime minister, accuse the government of lacking legitimacy because it came to power in a 2008 parliamentary vote after a court ruling removed Thaksin's allies.
Demonstrators have gathered in Bangkok since March 14, but Saturday's protests were some of the most confrontational.
Outside parliament, protesters cut through barbed wire and pushed past riot police before burning a copy of the constitution and then retreating.
The rallys were marred by three blasts that left eight people wounded, including five soldiers.
The largest explosion hit the government-run National Broadcasting Service of Thailand television station at 9.30pm local time (14:30 GMT), injuring three soldiers and a civilian guard, police and medical authorities said.
Earlier a grenade was thrown at army-run television station Channel 5, injuring four people - two civilians and two soldiers - and a small blast hit Thailand's customs department.
Thaksin, who was ousted in a coup in 2006 and lives in exile to avoid a jail sentence for corruption, regularly addresses his supporters by videolink and has urged them to intensify pressure on the government.
The red shirts' critics say the protesters are merely pawns serving Thaksin's ambitions to return to power.