The court ruled on Thursday that the military tribunal for a Guantanamo prisoner cannot proceed because it violates the Geneva Conventions.
The ruling, a rebuke to the administration and its aggressive anti-terrorism policies, was written by Justice John Paul Stevens, who said the proposed trials were illegal under US law.
The case focused on Salim Ahmed Hamdan, a Yemeni who worked as a bodyguard and driver for Osama bin Laden, the al-Qaeda chief.
Hamdan, 36, has spent four years in the US prison in Cuba. He faces a single count of conspiring against US citizens from 1996 to November 2001.
Two years ago, the court rejected Bush's claim to have the authority to seize and detain terrorism suspects and indefinitely deny them access to courts or lawyers.
In this follow-up case, the justices focused solely on the issue of trials for some of the men.
Stevens wrote for the court majority in the 5-3 decision that "the military commission convened to try Hamdan lacked power to proceed" because its structure and procedures violated the international agreement that covers treatment of prisoners of war, as well as US military laws.
Chief Justice John Roberts, named by Bush in September to the lead the court, was sidelined in the case because as an appeals court judge he had backed the government over Hamdan.
Thursday's ruling overturned that decision.
Tony Snow, the White House spokesman, said he would make no comment until lawyers had had a chance to review the decision.
Officials at the defence and justice departments were planning to issue statements later.
"The military commission convened to try Hamdan lacks power to proceed"
Justice John Paul Stevens
The Bush administration has hinted in recent weeks that it was prepared for the Supreme Court to set back plans for trying Guantanamo detainees.
The president told reporters: "I'd like to close Guantanamo. I also recognise that we're holding some people that are darn dangerous."
The court's ruling says nothing about whether the prison should be closed.
The prison at Guantanamo Bay, erected in the months after the September 11, 2001, attacks on the United States, has been a flashpoint for international criticism.
Hundreds of people suspected of ties to al-Qaeda and the Taliban, including some teenagers, have been seized by the US military and secretly shipped there since 2002.
Three detainees committed suicide there this month, using sheets and clothing to hang themselves.
The deaths touched off new scrutiny and criticism of the prison, along with fresh calls for its closing.