The Red Cross should also be given prompt access to anyone detained in connection with an armed conflict, said the UN panel in a 12-page report released on Friday.
"The committee is concerned by credible and uncontested information that the state party has seen fit to engage in the practice of detaining people secretly and in secret places for months and years on end," the report said.
Such practices also violate the rights of detainees' families, wrote the committee which held a two-day hearing last week on US compliance with the 1966 International Covenant on Civil and Political Rights.
The report said that the US "should only detain persons in places in which they can enjoy the full protection of the law."
"It should also grant prompt access by the International Committee of the Red Cross to any person detained in connection with an armed conflict," it said.
The panel said members of the US government who have used or approved banned interrogation techniques, such as use of stress positions or exposure to extreme heat or cold, should be punished.
The US government is expected to respond to the report later on Friday.
In May, the top UN anti-torture panel - a separate body from this committee - recommended closing Guantanamo Bay prison and criticized US use of secret prisons and the delivery of prisoners to foreign countries for questioning.
US officials say that Washington has always regarded the 1966 treaty as only applying to protection of human rights within the United States.