George Papandreou, the Pasok leader and former foreign minister, said: "We bear a great responsibility to change the course of the country ... We know that we can make it."
"We must all believe again we can succeed ... we cannot waste a single day," Papandreou, the 57-year-old son and grandson of former prime ministers, told jubilant supporters gathered outside the party headquarters.
Conservatives' leader resigns
Costas Karamanlis, the Greek prime minister, conceded defeat and resigned as New Democracy's leader following his party's worst ever electoral performance.
Late on Sunday, Karamanlis congratulated Papandreou and said he hoped "he succeeds in the great challenge of facing the economic situation".
Karamanlis, 53, had called the election halfway through his second four-year term, saying he needed a strong new mandate to tackle Greece's economic troubles.
But ahead of the vote, surveys suggested his party would lose power.
His conservatives have seen their popularity undermined by a faltering economy and a string of corruption scandals.
Al Jazeera's Barnaby Phillips, reporting from Athens, said the corruption scandals had severely damaged New Democracy and Sunday's result should be seen, above all, as a rejection of their rule.
"I think the over-riding sense of this election was 'throw the rascals out', by which I mean that New Democracy was seen as corrupt and incompetent - they had lost their touch," he said.
|Papandreou campaigned on a platform of taxing the rich and helping the poor [EPA]
Othon Anastasakis, the director of southeast European studies at Oxford University in the UK, said that voters would demand decisive action from the new government.
"Greece is used to voting for one or the other party ... There won't be a period of grace, they [Greek voters] will start criticising from the beginning," he told Al Jazeera.
Nikos Magginas, an economist with Greece's National Bank, said the main challenge for the new government was "to submit a credible budget and a realistic timetable for reducing fiscal imbalances".
Both parties had run election campaigns promising to tackle corruption and pull Greece out of economic crisis.
Papandreou promised a $4.36bn stimulus package, campaigning on a platform of taxing the rich and helping the poor, Karamanlis called for two years of austerity.
The vote was the third face-off between Papandreou, a US-born politician, and Karamanlis, heirs to two of Greece's most powerful political dynasties.