The human rights organisation said interviews with Salah Nasser Salim Ali, 27, and Muhammad Faraj Ahmed Bashmilah, 37, indicated they were victims of "the US administration's policy of secret detentions around the world".
"Their testimony will hopefully shed light on US detention centres just as sinister, yet less well-known, than Guantanamo," said William Schulz, executive director of Amnesty International USA.
Yemeni authorities said the men had been transferred from the US detention facility at Guantanamo Bay in Cuba about three months ago at the request of US officials, but Amnesty said separate interviews with the Yemeni detainees painted a different picture.
Not at Guantanamo
Amnesty said the men could not have been held at Guantanamo based on the length of travel they said they took from their initial confinement in Jordan and their description of underground detention facilities.
Human rights groups have railed against a practice known as rendition, in which detainees are transferred from one country to another, often in secret and without legal safeguards.
"We fear that [what] we have heard from these two men is just one small part of the much broader picture of US secret detentions around the world"
"The Department of Defence does not engage in renditions," said Air Force Major Mike Shavers, a spokesman for the department that operates the facility at Guantanamo Bay.
Shavers said records of who had been held in Guantanamo were confidential.
The men, who say they were tortured and beaten in four days of confinement in Jordan, said after their transfer they saw no daylight, had no chance of outside communication and were not told why they were detained,
"We fear that [what] we have heard from these two men is just one small part of the much broader picture of US secret detentions around the world," said Amnesty's Sharon Critoph, who interviewed the men in the Yemeni prison.
Amnesty, which says the US is holding detainees in secret in countries that could include Jordan, Pakistan, Egypt, Thailand and Afghanistan, called on the US government to disclose the whereabouts of secret detention facilities, disclose the identity of detainees and take immediate steps to end secret detentions.