The reshuffle also brought in new ministers for defence, finance and foreign affairs.
"Political experience as well as profound knowledge are required for our work," Kaoru Yosano, the new chief cabinet secretary told a news conference.
"I am firmly convinced that the prime minister has chosen the people who have great knowledge and wisdom."
The LDP is reeling after it lost control of Japan's upper house of parliament last month for the first time since the party was founded more than 50 years ago.
It managed to cling to power because it still controls the more powerful lower house.
The defeat came in the wake of a series of scandals involving members of Abe's outgoing cabinet.
Since he took power last September three of Abe's ministers have been forced to resign, and one committed suicide amid a financial scandal.
Abe himself, though, has refused to bow to calls from party backbenchers for him to stand down, despite approval ratings nearing just 20 per cent.
|Taro Aso says the party must win back |
a disillusioned public [EPA]
Speaking after his transfer to the post of LDP secretary-general, Aso said the party had to win back a disillusioned public but faced tough times ahead with one chamber controlled by the centre-left opposition.
"The major task for us, the Liberal Democratic party, is to restore people's trust in the party and show firm measures to cope with people's concerns over the future," he said.
In other key appointments, Fukushiro Nukaga, a defence minister under Koizumi, has been appointed to head the finance ministry of the world's second largest economy, while former foreign minister Masahiko Komura takes over at defence.