The politicians have yet to decide on a name for their new party, which they plan to launch on December 16, with general elections expected a few months later.
Zuma, after brushing off the threat posed by the splinter group, has been criss-crossing the country to shore up the party's base.
At a rally with 15,000 people in a Soweto stadium on Sunday, he insisted that the ANC would defeat the new party in next year's elections.
"We can't wait for them to form their party so that we can engage them in debate, not in anger," Zuma told his cheering supporters dressed in T-shirts bearing his image.
|Zuma began campaigning in the face of the threat from the splinter group [AFP]
"The ANC is still the party it was in the old days.
"We are going to win the upcoming election with an overwhelming majority, as we have done in previous years."
Al Jazeera's Zeina Awad, reporting from Soweto, met Younis, an ANC supporter, who said that the attempt to form a new party will not work.
"There will not be another organisation that can empower us. We have already chosen Zuma, and this party will not gain widespread support," Younis said.
Adam Habib, vice-chancellor of the University of Johannesburg, told Al Jazeera it was still early to determine the challenge the breakway faction would pose to the ANC in a general election.
"[But] they could constitute a serious threat since the party has attracted a non-racial base ... and that suggests that if they were to compete, they would be competing for the home base of the ANC," he said.
"By that fact they could truly have a significant impact on South Africa's political system."
The ANC won more than two-thirds of the votes in the last election.
The main opposition Democratic Alliance, perceived as a party for whites, poses little serious threat.
Not another party
Shilowa told The Sunday Independent newspaper in an interview that the party was not trying to become a new opposition.
"We want to be in the government in 2009. We are very clear that we are not doing this just to be another opposition," he said.
"It is about believing in ourselves and about South Africans believing that this new formation we are going to put in place will contest for political power, not just participate in the elections," he told the paper.
Mbeki has not given his support to the new movement, but told Zuma in a letter leaked to the press last week that he would not campaign for the ANC ahead of the polls.
Zuma is popular with poor South Africans, who make up 43 per cent of the population, and has strong ties with the country's powerful labour unions.
"Zuma understands the needs of the people," Soweto resident Malose Zondi said at the ANC rally.
"Those who want to break away from the ANC are charlatans who are out of touch with the needs of the people."