Carter said on Tuesday that a great deal of opposition to Obama by Republican politicians and conservative groups was based on racism.
"I think that an overwhelming proportion of the intensely demonstrated animosity toward President Barack Obama is based on the fact that he is a black man, he's African-American," Carter told NBC television.
Obama was last week accused of being a liar by a Republican legislator during a speech on healthcare reform to the US congress, while conservatives have accused him of pursuing a socialist agenda.
"I live in the South, and I have seen the South come a long way," Carter said in the television interview.
"But that racism inclination still exists, and I think it has bubbled up to the surface because of a belief among many white people, not just in the South but across the country, that African-Americans are not qualified to lead this great country.
"It is an abominable circumstance, and grieves me and concerns me deeply."
Reverend Graylan Hagler, a pastor and anti-racism activist, told Al Jazeera that racism was an endemic problem despite the election of Obama as the country's first black president.
"The president is not only the head of government, he is the head of state ... and no matter who he is or how much we disagree with his policies, the president should be treated with respect"
former US president
"One of the things that the president is trying to do is to reason with people and deal with the intellect, but racism is such an emotional subject in the US that it continues to dwell on that base level," he said.
"[In the US] we really do not know each other; we exist on separate sides of the tracks, separate neighbourhoods - and the fact is that there is a deep suspicion. A lot of white Americans feel that the country has slipped away from them."
Several Democratic politicians have expressed unease at the tone that some of Obama's opponents have used in recent rallies, particularly those addressing the future of US healthcare.
Some conservative supporters have been seen at town hall meetings on healthcare reform with guns, while Democrats arguing the merits of Obama's healthcare package have been heckled.
"There's something loose in the land, an ugliness and hatred directed toward Barack Obama, the nation's first African-American president, that takes the breath away," Colbert King, a columnist for The Washington Post, wrote.
Some Obama opponents have claimed that the charges of racism are an attempt by Democrats to stifle opposition to the administration's policies.
"It is an intimidation tactic. When you make that attack and call someone racist or homophobic it is a way to kind of silence them," Brendan Steinhauser, a co-ordinator for FreedomWorks, an opposition group, said.
"This movement is made up of people who oppose big government," said Steinhauser, describing the thousands of protesters who rallied in Washington against Obama last weekend.
Joe Wilson, the South Carolina Republican who yelled "You lie" during Obama's speech to congress last Wednesday, was formally rebuked on Tuesday in a House of Representatives vote.
The outburst came after Obama said that illegal immigrants would not be able to claim for federal subsidies to buy health insurance.
Carter said that Wilson's comment was "dastardly", and that it was evidence that racist views are still in evidence in US politics.
"The president is not only the head of government, he is the head of state," he said. "And no matter who he is or how much we disagree with his policies, the president should be treated with respect."