Douglas Wood, 63, an engineer who lives in California and is married to an American, was shown in a video on Friday with his head shaven, apparently pleading for his life.
Two masked men stood by with guns pointed towards the hostage in the video that carried the name of the group - Shura Council of the Mujahideen in Iraq.
"The family is shocked and horrified to hear of this ultimatum from Douglas' captors ... We do not believe Douglas' captivity or this ultimatum will make any difference to the policy of the Australian government," Wood's brother Malcolm said in a televised statement on Saturday.
"Douglas is a warm man of generous heart and spirit. His work is to help the people of Iraq towards a better life. We respect the people of Iraq, their patriotic spirit and their right to independence."
Appeal from Muslim leader
An Australian Muslim leader also appealed on Wood’s behalf. Mufti of Australia Sheikh Tajuddin Al-Hilali urged Wood's captors to release him, saying it would be in the interest of the Muslim community in Australia.
Al-Hilali said that not all Australians approve of Prime Minister John Howard's policy of sending troops to Iraq.
Australian Foreign Minister Alexander Downer stood firm on Canberra's refusal to give in to the Wood’s captors.
"We do not believe Douglas' captivity or this ultimatum will make any difference to the policy of the Australian government"
Douglas Wood's brother, Malcolm
"The important thing is that we don't look as though we're starting to cave in and give in to demands," he told Australian radio on Saturday.
"If you give in to demands ... more people eventually ... will be taken hostage and further demands made, so it's important we be strong and that, for the government's part, it just continues working at trying to get Douglas out."
Australia's Department of Foreign Affairs tells Australians not to travel to Iraq but further warned on Saturday that Australians in Iraq could be at risk of copy-cat kidnappings.
Downer has said that Wood may have been kidnapped from his Baghdad apartment up to two days before the first two-minute video was delivered to news agencies in Baghdad on Sunday.
That video showed Wood pleading at gunpoint for Australia, Britain and the United States to withdraw troops from Iraq.
Australia, a staunch US ally, was among the first to join the war on Iraq two years ago.
A new batch of 450 Australian troops are due to arrive in southern Iraq in the coming weeks to provide security and train the Iraqi army. They will take the total number of Australian troops in and around Iraq to about 1400.
"Now is not the time for any debate or controversy on Iraq policy. Now is the time to unite and stand shoulder to shoulder as Australia to try and secure Mr Wood's release"
Labour foreign affairs spokesman
Opinions polls in May last year showed that nearly two-thirds of Australians thought the war on Iraq was unjustified. Half of Australians thought it was not worth sending troops to Iraq, while 40 percent backed the conservative government's decision.
Australian Prime Minister John Howard won a fourth straight term at an election in October, crushing the centre-left opposition Labour, whose leader had vowed to bring Australian troops home by Christmas.
Labour pledged its unwavering support on Saturday for the government's effort to free Wood.
"Now is not the time for any debate or controversy on Iraq policy. Now is the time to unite and stand shoulder to shoulder as Australia to try and secure Mr Wood's release," Labour foreign affairs spokesman Kevin Rudd told reporters.