Trevor Phillips, head of the Commission for Racial Equality (CRE), said Islamaphobia and anti-semitism were being stoked up by leaders of fringe Islamist groups in the UK.
He said the sooner they were deported the better.
Phillips was referring to Abu Hamza al-Masri, leader of the Supporters of Sharia, and Omar Bakri Muhammad, of al-Muhajiroun.
The pair have been accused of creating divisions between Britain's Muslim community and wider society by expressing support for Usama bin Ladin.
"The faster [Home Secretary] David Blunkett can deport them the better," Mr Phillips told Reuters in an interview.
"Every time Abu Hamza or Sheikh Omar do their thing in front of the television cameras, Muslims suffer in dozens of places in this country.
"They claim to be speaking on behalf of Muslims but they do things in such a way that every time they speak, some Muslim somewhere else in Britain gets it in the neck."
"The Blair regime has found Trevor Phillips a useful, politically naive tool who is willing to do their dirty work of obfuscating their real and continuing war against Islam and Muslims"
Abu Hamza has been banned from preaching at north London's Finsbury Park Mosque.
He is currently fighting against the Home Secretary's decision to strip him of British citizenship and deport him to Egypt. It is thought Omar Bakri may suffer the same fate.
However, al-Muhajiroun have slammed Phillips's comments as "foolish" and "uncorrobarated".
Referring to the CRE the group said: "It is a reflection of the stance of this government quango organisation's close ties with the Blair regime and its fascist foreign policy in the Muslim world.
"The Blair regime and their media propagandists have been painting Islam and Muslims as fundamentalists, militants and extemists for years because of the Muslims rightful defence of their lives, property and honour in Palestine, Kashmir, Chechnya, Iraq and Bosnia."
Al-Muhajiroun added: "The Blair regime has found Trevor Phillips a useful, politically naive tool who is willing to do their dirty work of obfuscating their real and continuing war against Islam and Muslims."
And Abu Aziz, official spokesman for Abu Hamza, also denied his group was stoking Islamaphobia and anti-semitism.
"The faster [Home Secretary] David Blunkett can deport them the better... Every time Abu Hamza or Sheikh Omar do their thing in front of the television cameras, Muslims suffer in dozens of places in this country"
Commission for Racial Equality
He told Aljazeera.net: "Shiekh Abu Hamza is being punished for telling the truth. It seems anyone who speaks out against what is happening in Palestine and the Muslim world is censured.
"Abu Hamza is neither pro-West or pro-East, he is pro-Islam. He thanked the millions of westerners who marched against the Iraq war and said they put many governments in the Muslim world to shame."
He added: "He should not be deported. And to those who say he should, I say he will consider it of Husni Mubarak and King Fahd are deported to Britain. I think that's a fair swap."