Al-Marri is the only foreign national "enemy combatant" still in detention without charge in the US.
Indefinite military detention
"The government cannot subject al-Marri to indefinite military detention," the court's majority opinion said.
"For in the United States, the military cannot seize and imprison civilians, let alone imprison them indefinitely."
Jonathan Hafetz, litigation director of the Brennan Centre's Liberty and National Security Project, and the lead counsel for al-Marri, said the ruling was "a huge victory".
"The court soundly and rightly rejected the administration's attempt to treat the globe as a battlefield that is exempt from rule of law"
"The court soundly and rightly rejected the administration's attempt to treat the globe as a battlefield that is exempt from rule of law," he said.
The US justice department said it was "disappointed" at the ruling and said it would ask the court's full appeals panel of 13 judges to reconsider.
Citing "unrebutted evidence", it said al-Marri had trained at a "terrorist training camp" run by al-Qaeda in Afghanistan and also met Khaled Sheikh Mohammed, the alleged architect of the September 11, 2001 attacks in the US.
The justice department said that al-Marri then entered the US "to serve as an al-Qaeda sleeper agent and to explore methods of disrupting the US financial system".
The justice department added that George Bush, the US president, "intends to use all available tools at his disposal to protect Americans from further al-Qaeda attack, including the capture and detention of al-Qaeda agents who enter our borders".
"We accordingly intend to seek further review of today's decision."
Al-Marri arrived in the US on September 10, 2001 with his wife and five children.
He had a visa to study in Peoria, in the state of Illinois, where he had previously been studying.
He was arrested three months later under suspicion of credit-card fraud. But in June 2003, he was declared an "enemy combatant" by US authorities who ordered his transfer to a military prison in South Carolina.
On June 4, the military judges at Guantanamo separately dismissed charges against Omar Ahmed Khadr, a young al-Qaeda member from Canada, and Salim Ahmed Hamdan, a Yemeni said to be Osama bin Laden's driver.
The judges said they lacked jurisdiction because the detainees had not been classified as "unlawful enemy combatants" as stipulated by a 2006 law that established the military commissions.