US politicians are considering a non-binding resolution that would demand a formal acknowledgment and apology from the Japanese government for the brothels.
'Glossing over historical truth'
Abe's remarks on the issue last week drew angry responses from South Korea and the Philippines, both home to hundreds of women who say they were forced to work in the wartime brothels.
Abe told Japanese legislators he did not consider testimony given by surviving "comfort women" to the US congress to be conclusive evidence, insisting "there was no testimony that was based on any proof".
He said there was no evidence of coercion in the strict sense, such as kidnapping, but acknowledged that brokers procuring women otherwise forced the victims to work as prostitutes.
South Korea's foreign ministry accused Abe of "glossing over the historical truth".
In trying to deflect growing criticism on Monday, Yasuhisa Shiozaki, the Chief Cabinet Secretary, said Abe's remarks had been misinterpreted.
Shiozaki said the government was eager to resolve historical issues with its neighbours in a "forward-looking" manner.
Unapproved apology
Up to 200,000 mostly Korean and Chinese women are said to have served in Japanese military brothels throughout Asia in the 1930s and 1940s, according to claims backed by witnesses, victims and former Japanese soldiers.
These include accounts of abuse by the military, as well as kidnapping of women and girls for use in the brothels.
Prominent Japanese scholars and politicians have routinely denied direct military involvement or the use of force in rounding up the women, blaming private contractors for any abuses.
In 1993 Yohei Kono, the then chief cabinet secretary, apologised on behalf of the government for the wartime brothels, although his statement was not approved by the Japanese parliament.
The apology came after a Japanese journalist uncovered official defence documents showing the military had a direct hand in running the brothels, a role Tokyo had denied until that point.
Victims and their supporters have failed to claim official compensation from the Japanese government which set up a private fund in 1995 but has itself refused to provide money.
Opposition critics have hit out at Abe's comments, saying his refusal to acknowledge the issue endangers Japan's international standing as a nation supporting human rights.
"Unless Japan offers an apology ... I am afraid the international community will think Japan has not learned the lesson on human rights or from the war, which Japan started," Toshio Ogawa, opposition Democratic Party legislator, said.