Junichiro Koizumi told a secular official ceremony marking the 61st anniversary of Japan's surrender: "Our country caused huge damage and suffering to a number of countries, particularly people in Asia.
"On behalf of the Japanese people, I sincerely express condolences to the victims with our deep remorse.
"We are responsible for looking back at the past humbly and handing down the lessons of the terrible war to the next generations without forgetting."
About 7,000 people attended the ceremony.
Earlier on Tuesday, Koizumi prayed at the Yasukuni shrine, infuriating China and South Korea which associate the Shinto sanctuary with Japan's imperialist past.
In the South Korean capital Seoul, a foreign ministry official said South Korea would summon Tokyo's ambassador to protest.
The Shinto shrine honours Japanese leaders convicted as war criminals after the second world war along with 2.5million war dead. It is considered a symbol of Japan's past militarism in the two Asian countries, where memories of Japanese aggression run deep.
Koizumi, wearing a suit and looking solemn as he followed behind a Shinto priest clad in traditional robes, bowed before entering the inner shrine at Yasukuni's huge compound.
The pilgrimage, aired live by Japanese TV broadcasters, was the first by a Japanese prime minister on the August 15 anniversary since Yasuhiro Nakasone paid his respects there on that date in 1985.
Tokyo's ties with Beijing and Seoul have already deteriorated to their worst state in decades, in part because of Koizumi's past pilgrimages to the shrine.
China's foreign ministry said in a statement that Beijing had "strongly protested" against Koizumi's visit, which it said was "wrecking the political foundations of China-Japan relations".
The statement added: "Prime Minister Koizumi has constantly on historical issues hurt the feelings of the Chinese people and lost the confidence, not only of the international community, but also the Japanese people."
Koizumi's shrine visit has been
met with protests
While the United States has not publicly complained, experts say Washington is worried about Japan's growing isolation in the region and its deteriorating ties with China in particular.
A group of right-wingers attacked a van carrying opponents of the shrine visit, some throwing rocks and others banging on the vehicle with umbrellas. Riot police moved in and the van left.
Koizumi, who plans to step down in September after more than five years in office, promised during his campaign to become ruling party chief in 2001 that he would visit the shrine on the August 15 anniversary.
He has visited every year since, but never before on that date.
Koizumi says he goes to the shrine to pray for peace and honour those who sacrificed their lives for their country.
Critics argue his visits reflect Japan's failure to face up to its wartime past, including atrocities committed in Asia.
The shrine, which played a central role in the wartime state religion that helped mobilise the nation to fight in the name of a divine emperor, considers 14 wartime leaders convicted by an Allied tribunal as Class A war criminals to be "martyrs".