The organisation found the highest level of discrimination occured in the workplace, which the agency said was "worrying".
Morten Kjaerum, director of FRA, said: "Employment is a key part of the integration process. It is central to the contributions that migrants make to society, and to making such contributions visible.
"Discrimination may hamper the integration process."
'Access to justice'
The agency also found that Muslims aged 16 to 24 experienced more discrimination than other age groups, but that there were little differences in rates between men and women.
It said most believed discrimation occured on the grounds of ethnicity, and that discrimination on grounds of religion or belief was "very or fairly widespread".
The report, part of the first ever EU-wide survey on the experiences of immigrants and ethnic minorities, said discrimination against Muslims was similar to other minority groups in Europe.
Also in line with other minority groups, 79 per cent of Muslims responding to the survey said they do not report discriminatory incidents and cases of racist crime to any authority.
Kjaerum said more resources need to be allocated by governments for strengthening mechanisms to allow people to report racist incidents.
He said: "Public authorities have a responsibility to support the integration process. This includes making people more aware of their rights.
"All victims of racism must have access to justice - not only in theory, but also in practice."