The opposition newspaper El Nuevo Pais reported on Monday that 2.4 million signatures had been collected by Sunday. It added that the figure would reach more than three million.
The legal requirement for a referendum is 2.1 million.
"The referendum will show that at least two out of every three Venezuelans want the president to leave," editor Rafael Poleo said in a front page column.
According to Venezuelan law, the president can be removed after mid-term in office.
Chavez was ousted from power for two days, following a coup in April 2001 and by a general strike a year later.
Chavez cries foul
The embattled president accused the opposition of trying to remove him fraudulently. He said on state television that the petition drive was "shaping up to be an attempted mega-fraud that the Venezuelan people will not tolerate".
Chavez still commands immense
following among the poor
He pointed out to discrepancies between signatures and electoral rolls and said signers had been pressured.
William Lara, a former legislator and staunch Chavez supporter, called on the president's supporters to hold a rally.
Lara added that he would "inform international observers" of the "irregularities that clearly characterise a fraud on the part of the opposition."
The opposition had sent out vehicles to transport voters who had not yet cast ballots to the polls.
"Let's all go towards the mega-victory of peace, reconciliation and democracy," said Enrique Mendoza, leader of the Democratic Coordinator, the umbrella group that brings together the leading anti-Chavez forces.
Sixty thousand troops and 12,000 extra police were deployed across the country to prevent incidents. The opposition complained that some soldiers held up the petitioning with demands for identity papers.
Chavez enjoys massive support among poor Venezuelans for his populist policies, but is seen with distrust by the business community and many in the middle class who accuse him of ruining the economy.
The president announced last Friday that he would run for another six-year term in the 2006 election.