The High Court on Thursday found the Family Court did not have the jurisdiction to release three sisters, aged five, nine and 11, and two brothers, aged 13 and 15, or make any other order concerning the welfare of children held in detention centres.
"Under the constitution, the Family Court, as a federal court, only has jurisdiction that parliament confers," the court said in a statement.
A spokesman for acting Immigration Minister Gary Hardgrave said the government was now considering whether the children - who have been in the care of Catholic welfare agency Centacare since their release in August - should be returned to detention.
"The implications of this complex judgment will need to be assessed before making further comment. The current arrangements for the children will remain in place for the time being," said Hardgrave's spokesman.
The five children at the centre of this legal battle, who cannot be identified, had been in detention since arriving illegally in Australia with their mother in January 2001, claiming to be refugees from Afghanistan's Taliban government.
"The High Court's decision is unfortunate"
spokesman, A Just Australia
Although their father, who arrived in Australia in October 1999, was granted a temporary protection visa, the mother and children were refused visas on the basis they were from Pakistan and not Afghanistan and were awaiting deportation.
The father was later taken into detention after a review of his case.
But the family's case hit the spotlight in June 2002 when two of the boys escaped from a remote detention centre at Woomera in the Australia's harsh outback and spent three weeks on the run.
They were released in August the following year by the Family Court decision amid strong lobbying by pro-refugee groups.
"The High Court's decision is unfortunate," said Greg Barns, spokesman for refugee group A Just Australia. "The children in question have suffered enough as a result of government policy. It would be grossly inhumane to return them to detention."
Australia has one of the world's toughest immigration regimes, detaining boat people, illegal arrivals or anyone overstaying their visa in secure centres while their cases are handled - which can take years if appeals are involved.
International human rights groups, the United Nations and Australia's opposition Labour party have condemned the detention of asylum seeking children in these fenced camps, which have been plagued by riots, hunger strikes and suicide bids.