Former detainee Muhammad Khan and his legal team, headed by Muhammad Chaudhry, said writs had been sent to the US secretaries of defence, state and justice, via the American embassy.
The demand was for "mental suffering, financial losses and physical and religious victimisation", Chaudhry said.
Linda Cheatham, an official at the US embassy in Islamabad, confirmed the embassy had received a letter from Khan's lawyer.
"We received a letter and passed that to the Department of State, to Washington," she said.
Muhammad Khan was released in October from Camp X-ray, where the US government has been holding al-Qaida and Taliban suspects captured in Afghanistan and elsewhere.
Khan said he was caged in a tiny cell and kept in solitary confinement for weeks.
Chaudhry added he would give the government four weeks to respond. "If there is no response, then we will think to file a case either in a US court or in a Pakistani court."
Khan, who owns a saw mill, said he was seeking $10 million for mental suffering and victimization and a further $44,000 for lost income. He said he was in Afghanistan preaching when he was detained in Kunduz province.
Pakistani officials have said most of the 50 Pakistanis detained at Guantanamo Bay were not linked to the al-Qaida network of Usama bin Ladin, but only sixteen have been freed so far.
Some Pakistanis went to Afghanistan to fight alongside the Taliban. US forces detained dozens, holding them without charge and denying access to lawyers.