Thursday's defeat is the latest in a long line of defeats for the party's elder statesman and promises to shake up the country's political system.
Peres, the current party leader, had wanted Labour to remain as the junior partner in Prime Minister Ariel Sharon's governing coalition. Peretz has said the party should pull out of the coalition and work for early elections.
The loss cemented Peres' image as a perennial loser. While Peres, a Nobel peace laureate, is widely revered abroad, he has had trouble connecting with Israeli voters and failed in five previous elections for prime minister.
Earlier in the day, Peres said he suspected fraud in the party's primary and called for an internal investigation. Peres made his demand at a surprise news conference at 3.15am local time as the party was compiling votes from Wednesday's contest.
"It is illogical that in a number of villages, in which I know I had a majority, we dropped to seven votes," Peres said. "I expected a better evening."
A third candidate, Binyamin Ben-Eliezer, trailed with 17% and conceded the race. Opinion polls had forecast a decisive Peres victory.
Peres did not directly accuse Peretz of foul play, but said reports of wrongdoing by his opponent's supporters had to be checked.
"At this stage, we ask to check the complaints," he said. "We are turning to the legal institutions of the party to look into this."
Peretz did not immediately respond to the accusations.
As the votes poured in early Thursday, his supporters burst into loud cheers when he overtook Peres for the first time in the tally.
Analysts say the close contest is
sure to embarrass Peres
Peres led Labour into the government this year to shore up support for Sharon's withdrawal from the Gaza Strip.
The pullout divided Sharon's Likud Party, and with Labour's support, the plan could not have been carried out.
Peres, who won the 1994 Nobel Peace Prize for forging an interim peace agreement with the Palestinians, believes that remaining in the coalition will allow him to push forward with peace efforts.
Since the Gaza pullout, he has led negotiations with the Palestinians to resolve key disputes, such as control over Gaza's borders, and to help rebuild Gaza's shattered economy.
Labour has adopted more free-market economic policies in recent years, drawing criticism from its traditional supporters in the unions, working class and farming sectors.
"I think that there are a lot of people waiting for the moment that someone will fight for their right to make a living with respect, to grow old with respect," Peretz told Israel's Army Radio earlier in the day. "They see that the process is under way and I hope that the Labour Party members will not stop the social revolution."
"If Peres wins, the status quo continues. If Amir Peretz wins, there will be chaos in the leadership of the Labour Party"
Peretz said that if he wins, he would either negotiate an early election with Sharon or seek a parliamentary majority to topple the government. Likud remains divided in the wake of the Gaza pullout, and Sharon is unlikely to be able to maintain a parliamentary majority without Labour’s help.
"If Peres wins, the status quo continues," said political analyst Hanan Crystal. "If Amir Peretz wins, there will be chaos in the leadership of the Labour Party."
He said the party's leaders, who control eight government ministries, are content with being a junior coalition partner and in no hurry to head to elections. Sharon remains the country's most popular politician and Labour leaders believe they are in no position to unseat him, Crystal said.