The US defence secretary said on Wednesday: "We're at a point now when the security situation depends as much on the reconciliation process and on the strengthening of [government] ministries.
"It's as much a political task as anything."
He was speaking to reporters travelling with him on a visit that follows stops in Afghanistan and Tajikistan.
Rumsfeld said that because of political challenges, Iraq was not yet prepared to make decisions that might allow the United States to cut troop levels.
"We haven't gotten to that point," Rumsfeld said. He said that US commanders and Iraqi officials must first undertake a comprehensive review of needs on the ground before the US can consider reducing troop levels.
A 129,000-strong American force is serving in Iraq more than three years into the war in which about 2,500 US troops have died.
Security in Baghdad was among the topics the US defence secretary said he would discuss with commanders and Iraqi officials.
But Rumsfeld said he did not plan to discuss a series of recent accusations and charges against US servicemen of murdering Iraqi civilians.
Those cases, including the murder of 24 unarmed civilians in Haditha and the rape of a teenager and murder of her family, have battered America's image and led senior Iraqi officials to question the immunity US troops have from Iraqi law.
Nuri al-Maliki, the Iraqi prime minister, for example, has called for independent inquiries into the Mahmudiya rape and murder case.
Rumsfeld, however, said the cases would be handled by US military officials.