Howard, 68, had trailed in opinion polls all year with some forecasting a landslide victory for Labor, but surveys in the final days of the campaign said the contest was close.
Dan Nolan, Al Jazeera's correspondent in Sydney, said that Rudd and his Labor party had run a strong campaign and there have been comparisons with Tony Blair's campaign that ended a long period of conservative rule in the UK in the 1990s.
Nolan said: "All the election analysts say Australia will have a new prime minister."
Howard is a close ally of George Bush, the US president, and had made a commitment to keep Australian troops in Iraq if re-elected.
He also offered voters $29 billion in tax cuts, but few new policies.
In contrast, the Labor leader Rudd pledged to withdraw combat troops from Iraq and sign the Kyoto Protocol on global warming.
The 50-year-old former diplomat who speaks Mandarin would also be expected to forge closer ties with China and other Asian nations.
'Old and tired'
|Howard had made a commitment to keep|
Australian troops in Iraq if re-elected
Labor needs to win an extra 16 seats to take office.
Howard could also face the humiliation of losing his parliamentary seat of 33-years after new constituency boundaries turned his Sydney stronghold into a marginal seat.
Rudd, 50, has offered voters a generational change, saying Howard was too old and tired to lead Australia.
"I offer Australia new leadership for the future, a positive plan for the future because Mr Howard's government's best days now lay behind it," Rudd said on Friday.
"Mr Howard has gone stale in his government's approach to the future."
13.5 million people are eligible to vote from Australia's remote outback to the Antarctic.
Source: Al Jazeera and agencies