Japanese media have reported that Fukuda is likely to retain most of the ministers from Abe's cabinet, which was reshuffled just last month in an attempt by the outgoing premier to maintain his grip on power.
Fukuda said Taro Aso, the former foreign minister, who had been Fukuda's sole rival in the LDP leadership race, had declined an offer of a cabinet post.
Fukuda's first order of business will be pushing through parliament an extension to an Indian ocean naval mission in support of US-led forces in Afghanistan in a move the opposition has vowed to defeat.
Japanese tankers have been refuelling coalition ships in the Indian ocean since 2001, and the US – Japan's top ally and protector - has been pushing for an extension of the operation.
Earlier Shinzo Abe, the former prime minister, emerged from a Tokyo hospital, where he has been receiving treatment for a stress-related stomach illness, to officially dissolve his cabinet.
"I want to extend my apologies to the people for not being able to complete my duties," he said in a statement read to reporters following the cabinet's final meeting.
Abe resignation as Japan's Prime Minister on September 12.
He had faced harsh criticism for abruptly resigning amid the parliamentary battle over the extension of the country's naval mission.