The protest was the second since the alleged incident.
Demonstrators marched through Athens on Friday chanting "Allah is great" and carrying banners proclaiming "Hands off immigrants".
"We want the officer or officers involved to be prosecuted, and the government to issue an apology," Manala Mohammed, a Syrian national, told the Associated Press news agency.
"We want people to show us respect."
The protest in the city's Omonia square was organised by leftist, immigrant and anti-racism groups, which have previously complained of police brutality against immigrants.
Vasso Akrivou, a member of the Expel Racism group, told the AFP news agency: "Immigrants are outraged. The incident on Wednesday was the straw that broke the camel's back."
Police have said they will investigate the reports.
The Muslim Union, representing thousands of immigrants in Athens, said it had filed a case against the unidentified policeman.
"Police told us they need more time for the internal investigation so we went ahead and filed a suit ," Naim Elghandour, the union president, told the Reuters news agency.
Thousands of immigrants, many from Muslim countries, cross into Greece illegally every year seeking a better life in the West, but large numbers end up living in squalid, overcrowded apartments.
Trapped in legal limbo, most have no jobs and are often arrested for minor crimes.
On May 9, members of a rightist group attacked immigrants in Athens, sending at least three to hospital.
Rights groups accuse Greece, which is predominantly Orthodox Christian, of not doing enough to protect immigrants.