"In the present situation, it is difficult to push ahead with effective policies that win the support and trust of the public. I have decided that we need a change in this situation."
In recent months Abe has seen his approval ratings plummet to just 30 per cent.
"Japan needs a new leader to fight against terrorism," he said. "The people need a leader whom they can support and trust."
On Sunday, Abe had indicated that he would step down if he failed win backing from politicians for an extension to a Japanese naval mission supporting US-led operations in Afghanistan.
That extension seemed to be in question after the LDP suffered a bruising defeat in elections to the upper house of parliament in July.
Abe also suggested that his resignation would aid bipartisan passage of the bill extending the naval mission, citing the refusal of Ichiro Ozawa, head of the opposition Democratic Party of Japan, to meet him.
"The people need a leader whom they can support and trust"
He cited that and the July election defeat as leading reasons behind his decision to quit.
In the wake of the July's vote Abe reshuffled his cabinet in a failed bid to recover public support.
At the time Abe said he would not step down to take responsibility for his party's poor performance.
Stock market nerves
Japan's stock market was unnerved by Wednesday's surprise announcement, with the benchmark Nikkei 225 index ending the day down 0.5 per cent.
|Abe's sudden departure sparked |
nerves among investors [Reuters]
Investors said Abe's sudden departure meant further uncertainty over key budget and financial policies.
"This is absolutely negative to the stock market," Norihiro Fujito, senior investment strategist at Mitsubishi UFJ Securities told the Associated Press.
Others however said that Abe's departure from the premiership could prove positive in the longer term, if it clears the air and allows the LDP to get a fresh start.
Abe, Japan's first prime minister born after World War Two, took office last year with pledges to end legacies of defeat, boost Japan's global security profile and rewrite the US-imposed pacifist constitution.
But his authority was undermined by a raft of scandals, including massive mismanagement of the pension system - a sensitive issue in rapidly ageing Japan.
Speaking to reporters on Wednesday, Abe did not announce a date for leaving office, but said he had instructed LDP leaders to immediately begin looking for a replacement.
Kyodo News agency reported LDP officials were making arrangements to hold an election for a successor on September 19.
Former foreign minister Taro Aso is considered by observers to be a front-runner for the post.