The Labor leader said: "I want to thank all those Australians who have placed their trust in me and my team... I will never take their sacred trust for granted. This is a great responsibility... I stand ready to accept that responsibility."
Howard, who is likely to lose his own seat of Bennelong in Sydney, making him the first prime minister to do so in 78 years, described Labor's win as an "emphatic victory".
He said: "This is a great democracy and I want to wish Mr Rudd well. He assumes the mantle of the 26th prime minister of Australia.
"We bequeath to him a nation that is stronger and prouder and more prosperous than it was 11-and-a-half years ago."
He said there was "no prouder job in the world than being prime minister of Australia".
The election was fought mainly on domestic issues, with Labor capitalising on anger at workplace laws and rising interest rates which put home owners under financial pressure at a time when Australia's economy is booming.
Rudd, the 50-year-old Labor party leader, had presented himself as a new generation politician compared with Howard, who is 68.
Rudd promised to pull Australian combat troops out of Iraq and sign the Kyoto Protocol on climate change, further isolating Washington on both issues.
The former diplomat, who speaks Mandarin, is also be expected to forge closer ties with China and other Asian nations.
His message for change attracted a swing of more than five per cent across the nation from the previous election, locking in only the sixth change of government since World War Two.
Howard had offered voters $29 billion in tax cuts, but few new policies.
"I offer Australia new leadership for the future, a positive plan for the future because Mr Howard's government's best days now lay behind it," Rudd said on Friday.
"Mr Howard has gone stale in his government's approach to the future."
Election analyst Antony Green predicted Labor would win at least 80 seats in the 150-seat parliament, giving it a clear majority in its own right for the first time since it lost power to Howard in 1996.
Green said: "The victory is starting to become more emphatic and the Labor Party is picking up seats way beyond the 5 per cent swing required."
"We've all got goose bumps that finally we might have a leader who is passionate about fairness in this country.
"Finally, after 11 years, it's happening," Celeste Giese, 39, said at a football stadium in the northern city of Brisbane where Rudd was due to hold a victory party.
Dan Nolan, Al Jazeera's correspondent in Sydney, said that Rudd and his Labor party had run a strong campaign and there have been comparisons with Tony Blair's campaign that ended a long period of conservative rule in the UK in the 1990s.
Howard had trailed in opinion polls all year with some forecasting a landslide victory for Labor, but surveys in the final days of the campaign said the contest was close.
Howard was a close ally of George Bush, the US president, and had made a commitment to keep Australian troops in Iraq if re-elected.
Rudd has said he will withdraw Australian soldiers from the country.
A White House spokeswoman said that Bush looked forward to working with the new administration.
Emily Lawrimore said: "The United States and Australia have long been strong partners and allies and the president looks forward to working with this new government to continue our historic relationship."
"President Bush congratulates Kevin Rudd and the Australia Labor Party on their election victory."
Rudd had attracted criticism from Bush for the promised pullout, but in a speech on Saturday the Labor leader appeared to hold out an olive branch to Washington.
He said: "I extend our greetings tonight to our great friend and ally the United States."