Fox, a Christian peace activist, was abducted along with three colleagues in November 2005.
Announcing the discovery, the state department said it had no information on the condition of the other three hostages.
Noel Clay, a spokesman, said: "The state department continues to call for the unconditional release of all other hostages in Iraq."
Fox's family has been notified, he said, and "our heartfelt condolences go out to them".
Clay did not give any details as to the cause of death, but reports quoting an Iraqi interior ministry official said the body showed signs of beating on its back and had been shot.
Iraqi police found the victim in the al-Mansur neighbourhood of Baghdad at around 5pm on Thursday, the official said.
Recognising that they had found a Westerner, the police patrol contacted US forces who took charge of the body.
Separately a witness quoted by the AFP new agency described how the body of a thin man, wearing a track suit, was found.
The body had been wrapped in a blanket inside a plastic bag and placed on a rubbish dump, the witness said.
Clay said further forensic study would be carried out in the United States, but the FBI believed that the body was that of Fox.
On Tuesday, Aljazeera aired footage of the three other activists purportedly appealing to their governments to secure their release.
The hostages seen in the brief video dated 28 February were James Loney, 41 and Harmeet Singh Sooden, 32, both Canadians, and Norman Kember, 74, a Briton.
Allan Slater, a Canadian member of Christian Peacemaker Teams, said at the time that he was disturbed not to see Fox.
"We certainly are hopeful when we see three of our friends alive, but also it's very distressing that we didn't see Tom Fox, and I wouldn't want to hide that because I'm sure it's very distressful for Tom's family and friends as well," Slater told The Canadian Press from Baghdad.
The previously unknown Swords of Righteousness Brigades said they were responsibile for kidnapping the four workers, who disappeared on 26 November.
Fox (centre) was not seen in the
most recent hostage video
The four had not been heard from since a videotape aired by Aljazeera on 28 January, dated from a week before.
A statement accompanying that tape said the hostages would be killed unless all Iraqi prisoners were released from US and Iraqi prisons. No deadline was set.
Iraqi and Western security officials repeatedly warned the activists before their abduction that they were taking a grave risk by moving around Baghdad without bodyguards.
Christian Peacemaker Teams had been working in Iraq since October 2002, investigating allegations that US and Iraqi forces abused Iraqi detainees. Its teams host human rights conferences in conflict zones, promoting peaceful solutions.
In a statement after the discovery of Fox's body, the group repeated its call for the safe release of the other three hostages.
"In Tom's own words: 'We reject violence to punish anyone'"
Statement from Christian Peacemaker Teams
"In response to Tom's passing, we ask that everyone set aside inclinations to vilify or demonise others, no matter what they have done," the group said.
"In Tom's own words: 'We reject violence to punish anyone'."
In the three years since the US-led coalition invaded Iraq, insurgents have kidnapped at least 250 foreigners and killed at least 40 of them.
In one of the most high-profile cases, Jill Carroll, a freelance writer for The Christian Science Monitor, was kidnapped on 7 January in Baghdad.
Three videotapes of Carroll delivered by her kidnappers to Arab satellite television stations identified the group holding her as the Revenge Brigades.
Carroll's kidnappers have publicly demanded the release of all female detainees in Iraq.