"As I frequently say, I feel sympathy for the people who underwent hardships, and I apologise for the fact that they were placed in this situation at the time," Abe told the committee.
Abe claimed earlier this month that there was no proof that Japan's government or army kidnapped women to work as "comfort women", as the wartime sex slaves are known in Japan.
He said on Monday that he stood by a 1993 apology, known as the Kono Statement, that acknowledged official involvement in the brothels. But he said there would be no new apology even if US lawmakers adopted a resolution seeking one.
The prime minister's earlier comments denying official involvement in kidnapping women, mostly Asian, to work in the wartime brothels have angered Seoul and risked straining ties with Washington.
Michael Honda, a US congressman, has introduced a resolution calling for Japan to make an unambiguous apology for the suffering of the women.
No vote on the resolution, which Abe has criticised as full of errors, is expected by May, after Abe visits Washington for talks with George Bush, the US president.
Abe, who made his name as a politician by pushing for a resolution to a feud with North Korea over Japanese people kidnapped by the secretive communist state decades ago, has come in for criticism in the US media for his recent remarks on sex slaves.
The Asian Women's Fund, set up in 1995 and partly funded by the Japanese government, has provided the "comfort women" with two million yen [$17,000] each in compensation and medical support, along with a letter of apology signed by previous prime ministers.
But many of the women have refused to accept the money, saying the Japanese government itself should provide the compensation in recognition of its responsibility.