The judges on Monday announced it had been a close-run contest, but they eventually picked Banville for what they called his "masterly study of grief, memory and love recollected".
"I must thank the judges who are suddenly my best friends," he said after his shock win.
Bookmakers had made Banville a 7-1 outsider to land the coveted prize. British author Julian Barnes had been hot favourite to win the Booker at his third attempt.
Banville confessed he had been too drunk to take in the evening when first shortlisted for the Booker Prize back in 1989.
Asked how he would spend his $88,000 prize, he said it would be on "good works and strong drink".
The prize, founded in 1969, rewards the best book of the year from British, Irish and Commonwealth writers and guarantees the winner instant literary fame and a place in bestseller lists around the world.
"I must thank the judges who are suddenly my best friends"
Banville, 59, was the first Irish winner of the Booker since Roddy Doyle in 1993 with Paddy Clarke Ha Ha Ha.
Banville, reflecting on Ireland's troubled past relationship with England, said: "The English did many terrible things but one great thing you did was give us this extraordinary language ... and it works for us."
Chairman of the judges John Sutherland said it was a "closely contested last round in which the judges felt the level of the shortlisted novels was as high as it had ever been".
He said "discussions could have lasted for three weeks" but "there wasn't a row" and judges finally settled on Banville.
The Sea is the 14th novel by Banville. It tells the tale of widower Max Morden returning to the seaside village where he spent a formative childhood summer, plunging him back into his darkest memories.
Six authors were in the race for
the prestigeous award
Among the other contenders for the prize was former Booker winner Kazuo Ishiguro for his critically acclaimed Never Let Me Go about three clones bred to be organ donors. He was runner-up this time.
The Booker shortlist has been attacked in the past by critics who say the winners are all too often turgid tomes that appeal only to literary academics.
Graham Sharpe, spokesman for bookmakers William Hill, regretted the omission of three literary heavyweights from the 2005 shortlist - Salman Rushdie, JM Coetzee and Ian McEwan.
"We are left with a low-profile bunch of authors. This is an average rather than an extraordinary year," he said.