The Americans and their Afghan translators had been handed over to Afghan intelligence agents for interrogation while some of the Afghan prisoners were later released, officials said on Friday.
The US group, apparently fighting their private "war on terror" rented a house near Kabul's Intercontinental Hotel and lied to neighbours saying that they were trading in Afghan rugs and running an export company, an Afghan official said.
"After we learnt that the men the Americans were holding were innocent people, we released them," Interior Ministry spokesman Lutf Allah Mashal told reporters.
"The investigation is continuing, right now we only know that they were running a private jail," an intelligence official who requested anonymity, told reporters.
According to Afghan police, the US group allegedly detained men with long beards on the outskirts of Kabul and based their suspicions on dress code and appearances.
Detained Afghans were then interrogated in an attempt to have them confess to ties with the Taliban and/or al-Qaida.
However, a senior intelligence official, who asked not to be named, said that only three of those detained by the Americans had been released. The remainder were in the custody of Afghan intelligence agents.
Only one of the three Americans was carrying a US passport, he said.
On Thursday, the US State Department confirmed that the three men were indeed American citizens.
"The Americans were mainly detaining men with long beards on the outskirts of Kabul whom they suspected, based on only on their appearances"
Afghan interior ministry official
Department spokesman Richard Boucher identified two of them as Jonathan Idema and Brent Bennett but said the third man would not be identified because he had not signed a Privacy Act waiver.
"As far as what they are being held for and what charges might be proffered, I'd have to refer you to the Afghan authorities for that," Boucher said.
"Let me make clear, first of all, the US government does not employ or sponsor these men."
Earlier in the week, the US-led coalition force hunting alleged Taliban and Al-Qaida members in Afghanistan had warned in a press release that Idema had represented himself as an "American government and/or military official."
Afghan authorities have said that the three men were running a private jail in Kabul as part of their personal war against terror.
"Three foreigners who had formed a self-made group and were claiming their aims were to act against those carrying out terrorist attacks, have been arrested," Interior Ministry Ali Ahmed Jalali said on Thursday.
Human Rights Watch
"They did not have any legal connection with anyone and the United States was also chasing them," Jalali said.
More than a dozen Afghan soldiers were guarding the house which contained the jail and each room inside has been sealed off since it was raided on Monday.
The well-built, two-storey house surrounded by trees also contained a basement, which soldiers confirmed to reporters was used as the main jail.
Human rights watch issued a 59
page report about US torture
"They were keeping the detainees in the basement," one soldier said, adding that the doors to the cellar had been locked.
Some 20,000 US-led troops are in Afghanistan apparently to hunt and kill members of al-Qaida and the Taliban.
Human Rights Watch has said in a 59-page document that "US forces operating in Afghanistan have arbitrarily detained civilians, used excessive force during arrests of non-combatants, and mistreated detainees".
"The United States is setting a terrible example in Afghanistan on detention practices. Civilians are being held in a legal black hole, with no tribunals, no legal counsel, no family visits and no basic legal protections," Brad Adams, executive director of the Asia division of Human Rights Watch, recently said.