Yasushi Akashi arrived in Colombo late on Sunday as shelling continued overnight between the government and the Liberation Tigers of Tamil Eelam (LTTE).
Akashi met Sri Lanka's former chief peace negotiator Nimal Siripala de Silva, and was to hold talks with other Colombo-based political parties later in the day, government officials said.
Akashi is also expected to meet LTTE leaders during his six-day visit, although a meeting with the Tiger's reclusive commander, Velupillai Prabhakaran, is unlikely.
The envoy would "exchange views on the peace process and its future with leaders of political parties, international organisations and others," the Japanese embassy said in a statement.
Akashi’s arrival is the first of several by foreign envoys this week as the international community struggles to make sure planned peace talks on October 28-29 in Switzerland go ahead.
Norway, the main peace broker in Sri Lanka, was planning to send special envoy Jon Hanssen-Bauer on Tuesday to work out details for the talks, while the US Assistant Secretary of State, Richard Boucher, was also expected to arrive later this week.
More than 2,200 people have been killed in spiralling violence since December, according to official figures.
Both sides have accused each other of sporadic attacks since major clashes last week, including shelling and air strikes against villages.
At least four people were killed and two others wounded on the northern Jaffna peninsula on Saturday night when unknown gunmen attacked a bakery and a teacher training course being conducted by UNICEF at the Kopay Christian College, the pro-Tiger Tamilnet.com website said.
These deaths followed the murders of at least eight Sinhalese migrant workers shot over the weekend by suspected rebels near the frontline town of Vavuniya, 260 kilometres (160 miles) north of Colombo, police said.
Tamilnet also reported the abduction of at least 10 people in Jaffna since Friday, while the Civil Monitoring Committee (CMC), recently formed in Colombo by Tamil and Sinhala politicians, reported that at least 29 people remained missing.
"There are many acts of disappearances, abductions, killings and extortions ... that have gone unreported due to reasons of fear," the CMC said in a statement.