Abe took 464 of the 702 valid votes from the Liberal Democratic Party's politicians and party chapters, against 136 for Taro Aso, the foreign minister, and 102 for Sadakazu Tanigaki, the finance minister.
Abe, the current chief cabinet secretary, takes over the party leadership from Junichiro Koizumi, the outgoing prime minister, and is all but certain to be named as Japan's first prime minister to be born after the second world war on September 26.
Abe, who turns 52 on Thursday, would become the country's youngest prime minister and has promised to pursue growth while pushing the economic reforms begun by Koizumi, who took power in 2001.
Abe has said he will try to improve relations with China and South Korea which have been damaged by Koizumi's visits to a shrine commemorating Japan's war dead, including leaders convicted as war criminals.
Abe has stressed that better relations require efforts on all sides.
Abe has defended Koizumi's pilgrimages to the Yasukuni Shrine and has visited it himself in the past, but has not said if he will continue to pay his respects there as prime minister
He has been a passionate supporter of rewriting the country's pacifist constitution which was imposed after defeat in the second world war, and has pledged to forge even stronger security ties with Washington and put patriotism back in Japanese classrooms.
Abe became a household name four years ago for his tough stance in a feud with North Korea over Japanese citizens kidnapped by the North Koreans decades ago.
Koizumi, who was applauded when he cast his ballot, welcomed Abe's election. "From the bottom of my heart, I want to offer congratulations to Mr Shinzo Abe," he said.