The rally was the first demonstration since Costas Karamanlis, Greece's prime minister, reshuffled his cabinet on Wednesday.
The prime minister had vowed tougher policing to counter the "catastrophic violence" of last month's riots in Greece, which were sparked by the fatal shooting of a teenager by police on December 6.
The shooting of Alexis Grigoropoulos, 15, unleashed a wave of anger that degenerated into the worst riots Greece had seen in decades.
Hundreds of stores in several cities vandalised and dozens looted in the days following his death.
Police were frequently targeted during the height of the unrest and police stations in Athens and other cities were attacked with stones and officer's cars torched.
Since then there have been sporadic incidents in the capital.
On December 23 shots were fired at an anti-riot van in Athens, missing the 23 police on board but hitting the engine and two tyres.
A previously unknown group calling itself "Popular Action" claimed responsibility for the attack.
A policeman was seriously injured on Monday in a shooting in the capital that police said marked the return of the country's most dangerous extremist group.
A caller to a Greek television station claimed responsibility for the shooting in the name of a local far-left group Revolutionary Struggle, which figures on the European Union's list of terrorist organisations.
Police had earlier said that a submachine gun used in the early morning attack had been used in a previous action claimed by Revolutionary Struggle, which in 2007 had fired a rocket at the United States embassy.