Addressing an audience at the Council on Foreign Relations on Monday, Rumsfeld moved away from his previous statements that linked Hussein with the international Islamist organisation.
"I have seen the answer to that question migrate in the intelligence community over a period of a year in the most amazing way. To my knowledge, I have not seen any strong, hard evidence that links the two," he said.
Rumsfeld said there were differences in the intelligence community as to the nature of the relationship.
The nexus between al-Qaida and the Iraqi government was a key point in the US effort to persuade the world that Saddam Hussein had to be dealt with after the 11 September 2001 attacks on the United States.
In September 2002, Rumsfeld said the US had "credible" information that al-Qaida and Iraq had discussed safe havens and non-aggression agreements, and that its leaders had sought contacts in Iraq who could help them acquire weapons of mass destruction.
Rumsfeld said he had in the past
relied on the CIA for his data
But in his comments on Monday, Rumsfeld said he had relied on the CIA for his information in the past and appeared to blame the intelligence reporting for the way the relationship between al-Qaida and Saddam was portrayed.
"I just read an intelligence report recently about one person who's connected to al-Qaida who was in and out of Iraq. And it is the most tortured description of why he might have had a relationship and why he might not have had a relationship," he said.
"It may have been something that was not representative of a hard linkage," he said.