'Where was congress?'
Armitage said members of US congress were as much to blame as Bush administration officials for failing to perform adequate oversight of US detentions, interrogations and "renditions" - the transfer of suspects to overseas prisons with different rules from the US.
"I don't think members of the senate particularly wanted to look into these things and they might have to look themselves in the mirror," he said.
"Where were they? They weren't doing their job, they were AWOL [absent without leave]."
The CIA has admitted to waterboarding three suspects at the US prison at Guantanamo Bay in Cuba, including Khalid Sheikh Mohammed, the alleged mastermind of the September 11, 2001 attacks against the US.
Armitage also said that no one in the state department knew prisoners were being abused until the Abu Ghraib prison scandal emerged in 2004.
Armitage left the Bush administration after George Bush was re-elected US president in November 2004.
He announced his resignation the day after his superior, Colin Powell, Bush's secretary of state, quit.
Barack Obama, Bush's successor, ended the CIA's harsh interrogation programme after entering office, limiting the agency to methods approved by the army's field manual.
He also approved the closure of the US prison facility at Guantanamo Bay in Cuba within a year.
Leon Panetta, the director of the CIA, told congress last week that secret overseas prisons where CIA prisoners were waterboarded and interrogated by other harsh means were being closed down.
But he said he had no plans to prosecute CIA employees for their roles in the "black sites" deemed legal by their government superiors.
The first episode of Fault Lines with Avi Lewis, which investigates the US government's policies on the so-called war on terror, airs on Thursday April 16 at 0600 GMT.