Zebari said his war-torn country had made progress on improving security, but added that it faced a crucial period ahead of the 15 December parliamentary election.
"The difficult part has gone in my view. We're very close to reaching a more stable form of government and of security," Zebari told a news conference on Friday following a meeting with Japanese Prime Minister Junichiro Koizumi.
"Now, any premature withdrawal will send the wrong message to the terrorists, to the opposition ... that this coalition is fracturing and running, that their policies and strategies of undermining this process are winning."
He said he could not give a time frame for when the coalition forces would be able to leave Iraq, but added that it would be at least months after the new Iraqi government was formed.
US defence officials said this week that the Pentagon had planned to reduce the number of US troops in Iraq, currently 155,000, to about 138,000 after next month's election. This could be reduced further to about 100,000 next summer if conditions allowed.
Zebari (L) says Iraq is close to
achieving a stable government
Zebari said Koizumi was aware of the need for the international community to remain committed to Iraq, and added he believed Tokyo would decide in the next "few days or few weeks" on whether to extend the mandate of its troops deployed in Iraq.
Koizumi later told reporters that Tokyo would consider what steps it could take to help out the Iraqi people, but did not say whether it would keep its military.
"I would like to continue to think about what kind of aid Japan can provide for Iraq's country-building," Koizumi said.
Japan has sent some 550 ground troops to Samawa in southern Iraq to provide humanitarian and reconstruction aid, but the mission expires on 14 December.
Koizumi will not say if Japanese
troops wil stay in Iraq
While the troops do not take part in operations to maintain security, the deployment is the country's first significant overseas military mission since World War Two.
The dispatch, first approved in 2003, won praise from Washington, but is opposed by the majority of Japanese. In a Mainichi newspaper poll published in October, 77% of those surveyed said they were opposed to an extension.
The Asahi newspaper reported this month that Japan was considering withdrawing its troops from Iraq starting from the first half of next year and completing the process by September.