Zuma, an ANC heavyweight once seen as the frontrunner to succeed President Thabo Mbeki in 2009, was charged with rape on Tuesday following allegations made by an unnamed 31-year-old friend.
Zuma has maintained his innocence.
The ANC's National Working Committee said in a statement on Wednesday that Zuma will "not act or pronounce in the capacity of deputy president of the ANC for the duration of this trial".
Zuma had previously said he would quit junior party posts but would not step down as party deputy president.
The ANC called on "all members of the ANC and its alliance partners … to allow the law to take its course, to respect the rights and dignity of those involved, and to exercise discipline, patience and restraint".
The opposition has demanded his sacking over the rape charges which come on top of corruption charges that led to Mbeki sacking him as his deputy in June.
Meanwhile, the South African media lashed out at Zuma's secret court appearance, saying the news blackout showed a contempt for democracy.
The case was not put on the court roll, and Zuma - who is also facing two charges of corruption in a separate case which cost him his job in June - appeared an hour before the court opened.
"What's so special about Zuma? VIP treatment in court case"
South African daily
Journalists and photographers were barred from covering his appearance.
"Is Jacob Zuma above the law?," asked the Johannesburg-based daily The Star in an editorial.
"This chain of events amounts to treating our democracy with nothing but contempt," The Star concluded.
Another Johannesburg daily, The Citizen, headlined its front page story: "What's so special about Zuma? VIP treatment in court case", quoting lawyers saying the preferential treatment was a "sad day for justice".
The South African National Editors Forum (SANEF) issued a statement slamming what it described as a breach of media freedom.
"Journalists... are the eyes and ears of the public. The media is a force for good in society and an essential element of democracy - it should be allowed to do its job unhindered," the media organisation said.
National police spokeswoman Sally de Beer declined to comment on the controversy. "We've taken note and that's all I can say," she told AFP.