The protesters defied government orders declaring the area off-limits, vowing to push ahead with their demands for Abhisit to step down and call new elections.
"We will take it to the next level," Nattawut Saikua, a leader of the protest movement known formally as the United Front for Democracy Against Dictatorship, told the crowd.
"We can't say what we will do … we won't give the government chance to prepare."
A day earlier a grenade attack in the capital injured two policemen outside the headquarters of Abhisit's Democrat party.
The prime minister has been under pressure from Bangkok's elites, the middle-class and even his own government to use force to remove the protesters and restore order in the city.
The government on Tuesday obtained a court order to arrest protest leaders, but said they could only be detained during an act of speaking to crowds.
In recent days the demonstrators - some of whom turned have turned rowdy - blocked major roads in the capital and pushed through lines of government forces.
Abhisit has defended his stance in dealing with the protesters, saying the government remains committed "to enforce the law and solve the problem", but prefers to avoid violence.
On Tuesday afternoon he interrupted programming on all channels with a brief television address, saying his government "eased our security measures to ensure that no confrontation would spiral out of control".
"We are confident in using the law to resolve the problem and move things forward. I know that many of you who would like to see things set straight and the rules of law respected are discouraged," he said.
"But the current fragile situation, involving a large number of misinformed people, demands careful manoeuvring. We need to plan and implement everything to the last detail and with thorough care."
"The current fragile situation, involving a large number of misinformed people, demands careful manoeuvring."
Thai prime minister
The red shirts see Abhisit, an Oxford-trained economist, as a stooge controlled by the unelected elite establishment and military.
They say Abhisit - who came to power in a 2008 parliamentary vote after the courts dissolved a pro-Thaksin party in government at the time - should call an election and let the people choose their government.
The anti-government protesters have vowed to honour the outcome of a fresh election.
Charnvit Kasetsiri, one of the country's most prominent historians, called the situation "a game of brinkmanship".
"It's about who's going to blink or make the first mistake, and whoever makes the first mistake will inevitably lose."
In the last five weeks, foreign investors have pumped more than $1.6bn into Thailand's stock market, which is up more than 80 per cent over the past year - Asia's third-best performer.
| There are concerns a prolonged standoff will scare investors away [GALLO/GETTY]
But there is concern the political crisis could squeeze longer-term foreign direct investment (FDI), which has been volatile since the crisis began with a 2006 military coup that ousted the twice-elected Thaksin, following allegations of corruption.
Roberto Herrera-Lim, an analyst at the Eurasia Group consultancy, said that for the first time since the coup, foreign investors are "really taking a hard look at Thailand long-term because of this".
"They see that the conflict is much deeper than the elite conflict of the past."
The anti-government rally has sustained crowds of tens of thousands since it began more than three weeks ago, protesting on a platform of "double standards" and marginalisation in a country with one of the biggest rich-poor gaps in Asia.
Many of the protesters are loyal to Thaksin, seeing him as Thailand's first and only prime minister who looked out for the poor.
Thaksin, a telecommunications tycoon, has fled into exile to avoid a jail term for graft, a court judgement he has called politically motivated.