Einars Repse hailed the decision as one of the one of the three most important decisions in the ex-Soviet state's history.
The Latvian PM said that EU entry ranked among gaining independence during the inter-war period and then regaining it after the fall of Soviet Union in 1991.
"The third (came) today with the decision to join the European Union," he told reporters.
An LTV state television exit poll released shortly after polls closed at 10pm (1900 GMT) gave EU supporters 69 percent and the "no" side 31 percent.
Early results were even more emphatic, with "yes" voters ahead by 72 to 28 percent after about one tenth of the count.
Last to agree
A final "yes" will mark a success for the EU enlargement from 15 to 25 and give Brussels something to celebrate after Sweden rejected the euro last weekend.
Malta, Slovenia, Hungary, Lithuania, Slovakia, Poland, the Czech Republic and Estonia have already voted to join the EU. The 10th nation to join, Cyprus, is not holding a referendum.
"This poll reflects the current feeling about the actual outcome," Repse said at his own pro-EU party while guests were toasting and congratulating each other on the win in the ex-Soviet republic of 2.3 million people.
Supporters believe joining the EU will secure freedom, democracy and growth in the poorest of the accession countries.
But the atmosphere soon soured when a simmering dispute within the ruling four-party, right-wing coalition broke out into the open and threatened to cause a split.
Political analysts said the coalition may well break up after a cabinet meeting on Monday, a development which would not affect the country's EU policy but which would inevitably cast a cloud over the vote celebrations.
Many pro-Brussels voters hail EU membership as the crowning of ex-Soviet Latvia's "return to Europe" after more than a decade of painful reforms since regaining independence in 1991.
A full count is expected in the early hours of Sunday and preliminary data showed turnout at 69 percent two hours before the end of voting, comfortably clearing a 35 percent requirement to make the referendum binding.
Some Latvians distrust the European Union just as they did the Soviet Union, and regard Brussels as too remote to care for the interests of the small Baltic nation, which will be one of the minnows of the enlarged union from May 2004.
But supporters believe joining the EU will secure freedom, democracy and growth in the poorest of the accession countries, with an average salary of about $300 a month.
"Our horizons will be broadened," said Evita Gerkina, 26, at a pro-EU party in Riga's historic Old Town. "I don't think our language, culture and identity will disappear."