But the Obama administration has argued that the courts have no jurisdiction over non-US citizens being held in a foreign war zone.
The US justice department has said Guantanamo represents a different case, as the territory is "under the complete jurisdiction and control" of the United States, while Bagram is on sovereign Afghan territory in an "active theatre of war".
Held without charge
Devon Chaffee, an advocacy lawyer at Human Rights First in Washington DC, attended the court hearing for the Bagram prisoners.
She told Al Jazeera that judges at the appeals court are struggling with how to rule in the case without laying the groundwork for opening up the system to all non-US citizen detainees around the world.
"If they say that US courts have the ability to hear these three cases, are they opening up jurisdiction to all detainees held in US custody all over the world?
"So what you saw today was them trying to figure out where to potentially draw that line."
But she said the defence lawyers argued that the detainees represent a special case.
"They were brought into Bagram from outside of Afghanistan. They've been held for over six years in custody.
"They haven't being charged with a crime, and they've been given very little due process."
Bagram conditions criticised
More than 600 detainees are being held at Bagram prison in Afghanistan, though none of have been charged with a crime or put on trial.
International human rights organisations have long criticised conditions at the Bagram facility, where detainees have been held - many of them for six years.
Chaffee said the Obama administration has put in place some measures to reform policies at the detention centre.
"The administration has started to make some reforms with the procedures for detainees that are being held at Bagram, [but] it still has a long way to go into implementing those reforms," she said.
"Only two of the three petitioners in this particular case have been given the improved procedures that the Obama administration has put in place. One has only been reviewed under the less stringent procedures that were used by the Bush administration.
"So clearly the administration has a long way to go to improving its detainee policies in Afghanistan, as well as resolving the issues with regards to the Guantanamo detention facility."