The US president named Josh Bolten, an administration insider, to replace Card, who became the highest-ranking official to depart from the White House since even Republicans began to urge Bush to revitalise his struggling team.
In the Oval Office, Bush said Card had offered and he had accepted his resignation. He will leave the White House on 14 April.
"I have relied on Andy's wise counsel, his calm in crisis, his absolute integrity and his tireless commitment to public service," Bush said.
With job-approval poll ratings for Bush at an all-time low and Republicans worried about keeping control of the US Congress in elections in November, some in Bush's party have called for new blood in the White House staff.
The White House has come under increased criticism in the past year as it has faced a number of crises, including its handling of Hurricane Katrina, the war in Iraq and the recent flap over a proposed sale of some key US port operations to Dubai-owned company DP World.
Bolten became director of the Office of Management and Budget after serving as deputy chief of staff in the White House from 2001 to 2003.
He is the son of a CIA officer and is known as a quietly effective manager. He also rides a Harley-Davidson and plays in a White House rock band.