Abe becomes Japan's youngest prime minister at 52, and the first to be born after the end of the second world war.
Hidenao Nakagawa, who was named on Monday as secretary- general of the ruling Liberal Democratic Party, said: "Now the new era of Abe begins.
"Under Prime Minister Abe, we will unite to deliver the policy promises we made to the Japanese people," he told public broadcaster NHK.
Koizumi formally stood down as prime minister earlier on Tuesday. He was applauded by party supporters, including Abe, as he left his official residence.
In his farewell speech, Koizumi stressed the importance of economic reforms and cuts in government spending.
"When I first came to office, Japan's economy was in stagnation and there was a prevalence of pessimistic views," Koizumi said.
"Today there is a willingness to challenge the new times and a confidence that we can succeed if we try. The bud of reform is beginning to grow into a big tree."
Early reports suggest that Abe is surrounding himself with supporters of Koizumi's free-market reforms, in what appears to be a continuation of measures to further revive Japan's economy.
"Under Prime Minister Abe, we will unite to deliver the policy promises we made to the Japanese people"
Hidenao Nakagawa, secretary general of the ruling Liberal Democratic Party
On Monday, he gave the secretary-general post to his mentor Nakagawa, a fellow conservative.
Nakagawa has close connections to China and has said he will try to repair relations between the two countries after Koizumi's visits to the Yasukuni shrine.
Abe himself has not said if he will visit the shrine, which honours Japan's war dead and war criminals.
Meanwhile, appointments to Japan's new cabinet have been made.
Taro Aso, the foreign minister, who ran against Abe for the party leadership, has been reappointed in his exisiting post.
Fumio Kyuma, an ex-defence minister, takes up his former position again.
Koji Omi, a former minister for science and technology policy, has been appointed finance minister.
Hiroko Ota is to be minister of the economy.