"The people have chosen and their decision is respected. PASOK fought hard but it did not succeed'' said Papandreou, who heads the party founded by his father, Andreas Papandreou, the former prime minister.
|Harris Tsitsopoulous talks about student concerns|
The result saw thousands of New Democracy supporters thronging the streets of central Athens, honking horns, chanting slogans and waving blue party flags.
Karamanlis, who called the snap election six month early, said the vote showed Greece had given his party a "clear mandate" to rule.
He has pledged to push forward with economic reforms and an overhaul of higher education that have proven unpopular with unions and leftist groups, sparking mass demonstrations in recent months.
With nearly 75 per cent of the vote counted, official results showed Karamanlis' New Democracy party ahead with 43 per cent, with PASOK on 38.5 per cent.
Karamanlis needed at least 42 per cent of the vote to form a majority government in Greece's 300 seat parliament.
Karamanlis victory came despite criticism over his response to August's forest fires that killed more than 65 people, and a split in the right-wing vote.
"The electorate's message is that we, too, have responsibility for the state of the nation"
The fires, which broke out just a week after Karamanlis called the election, led to widespread anger with many accusing the government of incompetence and mismanaging the crisis.
But Al Jazeera's Athens correspondent Barnaby Phillips says many voters felt that the socialists who ruled Greece for several years also carried some of the blame for fires, and that enabled New Democracy to squeeze back into power.
"Most Greeks are not overwhelmed with this government, but did feel it was worth another chance," he said.
PASOK ruled Greece for most of the 1980s and 1990s, but lost elections to Karamanlis's party in 2004.
Conceding defeat, Papandreou said it was important for his party to listen to the message voters had sent.
"People sensed that the poor state of public administration, with the scandals and fires, had deeper causes," he said.
"The electorate's message is that we, too, have responsibility for the state of the nation."
In the end both New Democracy and PASOK lost ground smaller parties on the left and right making significant gains.
Among the winners is the nationalist LAOS party which is expected to become the first far-right party to win representation in the Greek parliament in 26 years.
LAOS ran on a tough anti-immigration platform and opposition to Turkey's efforts to join the European Union.
More than 9.8 million people were eligible to vote in Sunday's election, 500,000 of them young, first-time voters.
The vote was Greece's 12th parliamentary election since democracy was restored in 1974 following a military dictatorship.