Investigators have found no evidence linking Murat Kurnaz to Muslim fighters in Pakistan or Afghanistan, Uwe Picard, a prosecutor in Bremen, said on Tuesday.
Kurnaz, 24, was detained when he travelled to Pakistan in October 2001 by Pakistani authorities who then turned him over to US officials.
He said he was held in Afghanistan and then taken to the US facility in eastern Cuba in January 2002.
Lawyers were first able to visit him in 2004.
He was released in August and is back in his hometown of Bremen, northern Germany.
US officials maintain that Kurnaz was a member of al-Qaeda, based on what they said was "secret evidence".
But a later review of declassified evidence in the case showed no indication he was connected to al-Qaeda or the Taliban regime, or that he posed any specific threat, according to his lawyers.
Kurnaz went to court in the US to win his release, and a US federal judge ruled in 2005 that the evidence was insufficient to justify his detention.
Her ruling has stayed, however, while the US government appealed.
Prosecutors in Potsdam have also opened an investigation into as yet unidentified members of the German army in connection with Kurnaz's allegation that he was abused by two German service members while detained in Afghanistan, a spokesman said.
Kurnaz has said that one of the men he identified as being German grabbed him by the hair as he was lying prone and then let his head fall face first to the ground, to the amusement of US and other soldiers nearby.