It was the first victory for an emboldened opposition against Chavez after nine years of electoral defeats.
Adolfo Taylhardat, an opposition politician, told Al Jazeera the result was "a victory for Venezuelan democracy".
"We have defeated president Chavez's intention to change the type of state we are living in," he said.
"He wanted to turn Venezuela from a democracy to a socialist state in which he would have almost full power."
Chavez's 69 proposed changes would have allowed him to control Venezuela's foreign currency reserves, appoint loyalists over regional elected officials and censor the media if he declares an emergency.
It would also have allowed him to run for re-election indefinitely.
But students, rights groups, business lobbies, opposition parties, the Roman Catholic Church and former political allies all lined up against Chavez, calling the proposed changes authoritarian.
Even Chavez's usually loyal ex-wife voted against his reforms.
"This was a photo finish," Chavez said immediately after the vote, adding that his respect for the results proved that, unlike past Venezuelan governments, he respects the will of the people.
He said he would "continue in the battle to build socialism" and told his supporters: "Don't feel sad."
Chavez publicly congratulated the opposition and urged restraint from both sides.
"I ask all of you to go home, know how to handle your victory," he said. "You won it. I wouldn't have wanted that Pyrrhic victory."
Tensions had surged in recent weeks as university students led protests and occasionally clashed with police and pro-Chavez supporters.
George Ciccariello, an expert on Venezuela with the University of California at Berkeley in the US, told Al Jazeera there had been "a great deal of disinformation" about Chavez's campaign prior to the vote.
"There were rumours, there was pamphleteering, there was printing false copies of the reform proposal.
"That said, Chavez really took a hit on this in terms of his moderate supporters not turning up to vote."
Lucia Newman, Al Jazeera's correspondent in the Venezuelan capital Caracas, said the result was a blow for Chavez but he continued to be popular.
|Chavez had predicted the referendum vote|
would be another knockout to Bush [AFP]
"He is going to have to, at the very least, move forward on his socialist project much more slowly and with more caution," she said.
"He's going to have to listen more to his own supporters, because there had been hints for quite some time that they didn't want to go forward with this."
Newman said it was the "political aspects" of the proposed constitutional changes that had concerned voters.
"There were some proposals that people actually liked. Such as the 36-hour work week, the voting age being lowered to 16 years from 18, the fact that self-employed people would be guaranteed a pensions."