Officials with the envoy, Yasushi Akashi, had asked to meet with reclusive rebel leader Velupillai Prabhakaran on Tuesday, rebel spokesman Daya Master said by telephone from their stronghold of Kilinochchi.
"But we told him it won't be feasible, but our political wing leader will be available," he said late on Saturday.
Akashi arrived on Saturday and will meet in the capital, Colombo, with Mahinda Rajapakse, the president, during his four-day visit.
The Japanese embassy said he was also "seeking the possibility of" meeting with rebel leadership in the north.
"I would like to talk to as many people as possible to get a clear view of prospects of peace," Akashi said on his arrival.
Japan is Sri Lanka's largest aid donor and has taken an interest in building peace on the island nation, where violence has left more than 150 people dead since April, threatening a 2002 cease-fire brokered by Norway.
Peace talks planned for last month collapsed amid a spate of landmine attacks and unsolved killings. The rift widened further with a suicide attack by a suspected rebel on the army's Colombo headquarters and two days of government air strikes against Tamil rebel bases.
The rebels are demanding a homeland for the Tamil minority, claiming discrimination by the ethnic Sinhalese majority. The current government won an election last year with a pledge to maintain a single state.
On Saturday, a government soldier was shot and injured in Jaffna, a northern town rebel-held territory.
This visit marks Akashi's 12th to Sri Lanka since Tokyo named him as its envoy in October 2002, said Hideaki Hatanaka, a Japanese foreign ministry official.
In 2003, Akashi helped organise a donors' conference in Tokyo that received pledges for $4.5 billion, mainly for the reconstruction of Sri Lanka's war-ravaged northeast.