Barnaby Phillips, Al Jazeera's correspondent in the Greek capital, said the pensioners were opposed to the government measure.
"The people in this crowd say they are very hard up and they simply can't afford for their pensions to be reduced in any way," he said.
"This is just a sample of the kind of concern there is from many parts of Greek society and we can expect more protests, more strikes, in the weeks ahead.
"However, it is important to say that opinion polls at the moment still show a majority backing the prime minister ... as he tries to implement these austerity cuts."
The Adedy union, Greece's largest public sector union, has already called for a 24-hour strike on March 16 to protest against the measures.
The union, which has around 300,000 members, also announced a series of raillies to be held throughout March.
The main tax service union, which has a membership of about 15,000, said it would hold a 48-hour strike from March 8.
The union has already held three strikes in the past month.
Greece first came under fire from member EU states when it revealed in October that its deficit would be 12.7 per cent of GDP in 2009, over four times the eurozone's three per cent limit.
The government had already committed to a series of cuts in public sector spending and tax raises in a bid to restore investor confidence.
The country's ballooning budget deficit and its problems in capping it have shaken faith in the strength of the euro, with EU partners fearing that the unstable market will spread to other eurozone members that have big deficits, such as Spain and Portugal.
On Tuesday, in a dramatic speech to members of his ruling PASOK party, Papandreou had compared his country's fiscal crisis to a war and warned he would have to take harsh and possibly unfair measures.
Papandreou said all Greeks would have to accept painful sacrifices, and warned of "catastrophic" consequences unless the country can borrow on international markets at lower lending rates.
The prime minister is due to travel to Berlin on Friday to meet Angela Merkel, the German chancellor, who had demanded additional fiscal steps from Greece before considering any European financial safety net for the eurozone's weakest economy.