Junichiro Koizumi, who left Japan on Saturday, will head to Ghana after Ethiopia. He is the first sitting Japanese leader to visit the two countries.
"The Japanese prime minister's visit to Addis Ababa underlines the importance of Africa for Japan and Japan's interest in Africa," a statement from Japan's embassy in Ethiopia said.
His week-long trip comes amid strained relations between Japan and China, which has been seeking to increase its own influence in the developing world.
Wooing Africa is a key part of Tokyo's strategy to win a permanent seat on the United Nations Security Council - a Japanese foreign policy goal that China has opposed.
Koizumi, Japan's longest-serving premier in a generation, last visited the continent in September 2002 for a global summit on sustainable development in South Africa.
In Ethiopia he was to meet Meles Zenawi, the prime minister, on Sunday and then on Monday the head of the African Union commission, Alpha Oumar Konare, whose organisation is based in the Ethiopian capital.
The only other sitting Japanese leader to go to Africa was Koizumi's predecessor Yoshiro Mori, who travelled to Kenya, Nigeria and South Africa in 2001.
Japan, the world's second largest economy and a major aid donor, in 1993 launched an initiative to drum up international support for Africa.
At a summit of African leaders in Tokyo in 2003, Japan offered $1 billion to the continent over the following five years, while Koizumi vowed to triple aid to Africa in three years at last year's Group of Eight summit.
Even so, last year Japan's joint proposal with Brazil, Germany and India to enlarge the Security Council ended in vain after they failed to win full backing from the 53-member African Union.
Wealth and influence
Other Asian countries are also attempting to forge stronger ties with Africa, which still suffers from poverty and civil conflicts but has rich natural resources and often holds casting votes at international conferences.
Hu Jintao, the Chinese president, visited Morocco, Nigeria and Kenya in late April after Roh Moo-Hyun, the South Korean president, went to Egypt, Nigeria and Algeria in March.
China, the world's number-two energy consumer after the US, has been racing to secure dependable sources of energy to feed its rapidly expanding economy.
Hu Jintao is welcomed in Rabat
Beijing has drawn criticism for investing in Sudan despite pro-government militias' crackdown in Darfur province. In February, Angola, which has also enjoyed major Chinese attention, reportedly topped Saudi Arabia as China's leading oil supplier.
In a bid to take the lead over its Asian rivals, Japan has said it is shifting the destination of its foreign assistance from "just poor countries" to more strategic partners.
Japan, whose aid had focused on Asia, plans to terminate financial assistance to China by the time of the 2008 Beijing Olympics to symbolise the country's emergence.
In contrast, Japan's aid to Africa is increasing. Japan's grant-in-aid to Africa for the year to March 2005 rose 13.5% from the previous year to 45 billion yen ($390 million).
Loans, however, have remained extremely thin as Japan is hesitant to lend money to developing countries in the wake of a series of debt waivers demanded by them, notably African nations.