Imam Naji Ali, of Project Islamic HOPE, said on Wednesday a coalition of prominent Muslim organisations, imams and activists were fed up with Bush's "anti-Muslim" policies.
Ali said the Bush administration's foreign policy was responsible for the deaths of innocent Muslims overseas.
He said The White House appointment of the "Islamaphobic" Daniel Pipes to the US Institute of Peace was a slap in the face of the Muslim community.
He also criticised the Defense Department for inviting Reverend Franklin Graham, who had called Islam a "very evil and wicked religion", to deliver a speech at the Pentagon.
"President Bush's public statements have consistently portrayed Islam as a peaceful religion that has been hijacked by the forces of terrorism. But his official policies have done little to mark that distinction," said Ali.
"Muslim American leadership should send a message to President Bush that we will no longer suffer and be mistreated by his administration's unfair domestic and foreign policies... Let President Bush eat by himself at his Ramadan dinner."
Ali also accused some Muslim organisations who backed Bush for president in 2000 of being politically naive.
"Muslim American leadership should send a message to President Bush that we will no longer suffer and be mistreated by his administration's unfair domestic and foreign policies"
Imam Naji Ali,
Project Islamic HOPE
"It was the immigrant Muslim groups that backed him, but we figured his father was well known for his anti-Muslim policies and he would just continue on the same road."
He added: "Muslim groups who attend events like these don't have any influence over Bush anyway, so they should stop letting themselves be exploited.
"American Muslims should support candidates that serve our interests and, at the moment, it looks as though that will be a Democratic candidate at the next presidential election."
President Bush has always maintained the "war on terrorism" is not a war on Islam.
He has also lauded the contribution American Muslims have made to the United States - economically, culturally and politically.
The White House has held a Ramadan dinner for leading Muslim groups and activists for the last two years.
The meal has always been well attended and a boycott would be unprecedented.