The killing of Katsuhiko Oku, 45, and Masamori Inoue, 30, near Tikrit also prompted opposition parties and common citizens to come out against sending troops to Iraq.
"Even though this incident has taken place, Japan must not give in to terrorism," Koizumi told reporters.
"We will firmly carry out our responsibilities for humanitarian aid and reconstruction (in Iraq) as a member of the international community. There is no change in this."
The deaths were the first of Japanese in Iraq since the US-led war began in March and will increase pressure on Tokyo as it weighs a decision on sending troops to help rebuild the country.
The prime minister added that more attention would have to be paid to security. "We will have to determine the causes of this incident and take firmer defensive steps," he said.
"There are areas (of Iraq) where safety can be assured and areas where it cannot. We will have to consider this."
Opposition parties, meanwhile, blamed Koizumi for his government's insufficient security measures in Iraq, and urged him to abandon the planned troops dispatch.
"There is no doubt that people's confidence in safety there has now collapsed," Seiji Maehara, a lawmaker and senior official of the largest opposition Democratic Party of Japan, said.
It is "too early" to decide to send troops to Iraq, said Maehara, who is in charge of the party's security policy. "We will firmly oppose it."
The diplomats' deaths cut the number of Japanese embassy-related staff in Iraq to nine, but Tokyo had no plans at present to close the Baghdad embassy, Foreign Minister Yoriko Kawaguchi said.
The incident has led to common citizens too questioning the rationale of sending troops to the volatile region.
"The news was a shock to me," Kazuko Sakata, a 40 year-old owner of a cosmetic company, said. "The two people were too young."
Seven Spanish officers were kliled
on the same day near Baghdad
Sakata said he opposed sending troops to Iraq, believing "that a fight cannot be solved by another fight".
"I think Japan is just being manipulated by the United States and I don't think it's right."
Retsu Matoba, a 30-year-old businessman, said, "It was wrong for them to go to such an insecure area. I know it was reckless. The government should heighten security. It's dangerous."
But Masaaki Okamoto, a 70 year-old pensioner, said Japan as an ally of the United States would have to send troops sooner or later. "I know it involves danger but there is no other choice," he said.
"I think that nowadays, there is nowhere that is safe. Even if Japan doesn't send its Self-Defence Forces, there is no guarantee that Japan will be free from terrorist attacks."
A reconnaissance mission returned to Tokyo on Thursday and was expected to report that it was safe enough to send Japanese ground troops to the southern city of Samawa.