The president has faced criticism over allegations that he pushed for corruption charges to be brought against Jacob Zuma, the ANC leader and his political rival.
A court ruling, handed down on September 12, threw out charges of corruption against Zuma, saying the prosecuting team violated some of his procedural rights.
The government has denied interfering in the case.
Mbeki said the governmet "never compromised the right of the national prosecuting authority to decide whom it wished to prosecute or not to prosecute".
"This applies equally to the painful matter relating to the court proceedings against the president of the ANC, comrade Jacob Zuma," he said.
Mbeki's term had been due to expire in mid-2009.
Under the South African constitution, the president is appointed by parliament, which has been dominated by the ANC since the end of apartheid and the start of majority rule in May 1994.
Baleka Mbete, the speaker of parliament, is expected to be named the interim head of state until elections are held next year.
Zuma will likely run for presidency then, but is not eligible to replace Mbeki now because he is not a member of parliament.
Gwede Mantashe, the ANC's secretary-general, insisted the decision to ask Mbeki to stand down had been taken in the interests of party unity.
"This is not a punishment," he said. "We decided to take this decision in an effort to heal and reunite the ANC."
The ANC has been divided since Mbeki sacked Zuma as the deputy president, following corruption charges against him.
Al Jazeera's Jane Dutton, reporting from Cape Town, said: "Once Thabo Mbeki got rid of [Zuma] in 2005, the unioners turned against him, as well as the ANC Youth Group, who thought that Mbeki was undermining Zuma."
She said the criticism surrounding Zuma's trial earlier in the month was "what the ANC needed, the final straw to get rid of Thabo Mbeki".