Zuma's former financial adviser Schabir Shaik was convicted of corruption and fraud this month in a Durban High Court ruling that also implicated Zuma and said the pair's relationship was "generally corrupt", sparking calls for Zuma to resign.
"I've come to the conclusion that the circumstances dictate that in the interests of the honourable deputy president, the government and our young democratic system ... it will be best to release honourable Jacob Zuma from his responsibilities as deputy president of the republic and member of the cabinet," Mbeki told a joint session of parliament on Tuesday.
He added: "I wish to thank him for the service he has rendered as part of the executive, at national and provincial levels, sparing neither strength nor effort to ensure that, with each passing day, we build a better life for all South Africans."
Zuma, however, remains popular with the ruling African National Congress (ANC) rank and file as well as its trade union and communist allies.
He had refused to step down, saying he believed he had committed no crime and had not been tried in a court of law.
Shabir Shaik was jailed for 15
years in a ruling last week
Shaik was jailed for 15 years last week on charges of paying Zuma $189,400 in return for using his political influence to further Shaik's business interests.
The court also said he sought a bribe for Zuma from a French arms firm in return for protection from a government probe into an arms deal.
Mbeki appointed Zuma after elections in 1999.
Zuma, a Zulu, was key to ending years of bloodletting between the ANC and the mainly Zulu Inkatha Freedom Party.
Mbeki said he would announce Zuma's replacement in due course.