Jerzy Szmajdzinski, Polands minister of defence also stressed that the presence of US and Polish troops saved the archaeological site from looters.
Speaking on state radio, Szmajdzinski suggested that an investigation might be opened to determine "who damaged it, at what point in time and at what stage".
A British Museum report released on Saturday raised concerns that Babylon, one of the world's most important archaeological sites, had suffered from the troop presence.
"The condition of Babylon is neither the worst nor the best," Szmajdzinski said.
"Where there is war, where there is activity including man's intervention, there is always some sort of damage."
But, he said, if not for the US and Polish troops, "Babylon would have been looted, just like all museums in Iraq have been looted."
"Today ... we would be seeking and buying back bricks and fragments of walls," he said.
Poland, which commands the international security force in Iraq, took over the base at Babylon from the US Marines in 2003 and stationed troops there until the end of 2004.