A brief stint in power has now been extinguished for Japan's Democrats after a landslide victory returned the long-ruling conservatives to government.
When the Democratic Party of Japan (DPJ) ascended to governance in 2009, analysts predicted a "sea-change" in the country's politics. But with the Liberal Democratic Party's (LDP) strong showing in Sunday's election, Japan appears to have again embraced the centre-right values that have dominated the nation for much of the past six decades.
Before 2009, the LDP had been in power since 1955, barring a short 11-month period in opposition. Even then it was the largest party in the Japanese parliament, the Diet.
When two conservative parties merged to form the LDP in the mid-1950s under former Prime Minister Yoshida Shigeru, the influence of left-leaning parties was dramatically reduced in the Diet.
Angus Lockyer, a professor of Japanese history at the School of Oriental and African Studies, told Al Jazeera it became "what they call a 1.5 party system" - meaning the political landscape was designed for stability, not plurality.
"The disparate origins of the party came back to haunt them."
- Michael Penn, Shingetsu News Agency
The DPJ, however, was formed in 1998, committed to breaking LDP dominance and to establish Japan as a "genuine" two-party system.
The Democratic Party's 2009 victory promised to shift decision-making - in terms of drafting law and determining policy - away from the bureaucracy to the cabinet, and to reframe the nation's relationship with the United States, suggesting a watershed moment in Japan's post-war history.
Things, however, did not pan out that way.
Reining in the bureaucrats
In its three years in power, the DPJ has gone through three leaders, as well as frequent cabinet shuffles and re-shuffles. Ichiro Ozawa, the party's chief strategist and powerbroker, resigned along with 49 other members to form a rival party, and the leadership was criticised for its handling of the Fukushima nuclear disaster.
The party was also never able to make any serious inroads into limiting bureaucrats' powers or in recalibrating the US-Japan relationship.
Gerald Curtis, a political scientist at Columbia University and an expert on Japanese politics, said the inability of Yukio Hatoyama, the DPJ's first prime minister, to cling to power was crucial to the failure to fundamentally change Japanese politics.
Had Hatoyama been "more competent and had he remained in office for a few years, the party's plan to rein in the bureaucracy and create a powerful cabinet-led government might have borne fruit," he wrote.
So, what went wrong for Hatoyama? Under his leadership, the early signs of harnessing the bureaucrats at Kasumigaseki - home of Japan's government ministries - were promising. Hatoyama abolished the vice minister's meeting, the most senior-level policy meetings in the bureaucracy, and instead empowered political figures at the ministry level.
The changes meant DPJ politicians were now responsible for drafting law and developing their own policies, while Hatoyama was to lead and co-ordinate the process.
These radical institutional shifts to the most vital cog of a long-established system were not going to be easy to navigate. Side-lining bureaucrats not only meant dealing with a lack of policymaking knowhow, but also the loss of crucial institutional support and goodwill among a powerful body of people.
|US Marine Corps Air Station Futenma on Okinawa [Reuters]
However, Tomohiko Taniguchi, a professor at Keio University and a former spokesman for the Japanese Foreign Ministry, argued that Hatoyama's main problem was not that he attempted to marginalise the bureaucrats, but the "volatile" manner in which the DPJ pursued their "unachievable" policies.
A combination of seemingly inept leadership, inexperienced policymakers and an obstructive bureaucracy thwarting policy implementation meant the transition was a troubled one from the start.
Hatoyama also faced challenges from within his own party. The DPJ had been formed in a merger of small parties, along with reform-minded LDP defectors and former Socialist party members.
The loose threads within this patchwork party started to unravel as Hatoyama's competence on both domestic and foreign policy was questioned. The party was able to pull off an historic election victory but shortly after, "the disparate origins of the party came back to haunt them", Michael Penn, president of the Shingetsu News Agency, told Al Jazeera.
If the party, especially its leaders, had been able to stay united, it could have stood up to the pressure from the bureaucracy as well as the US, he added, but instead it suffered "disunity" and regular power struggles.
On top of domestic concerns, there was trouble on the foreign policy front, too. Hatoyama wanted a more "equal" relationship with the US and closer ties with important regional actors, especially China and South Korea.
Mark Selden, a senior research associate in the East Asia Program at Cornell University, told Al Jazeera that Hatoyama's vision required reducing Japan's dependence on the United States.
"He was met, however, by staunch opposition in the bureaucracy, notably the Foreign Ministry, but also METI [the powerful Ministry of Economy, Trade and Industry], and indeed across the board," said Selden.
The big test for the policy shift came with the decision over the planned relocation of the US Marine base Futenma on Okinawa. The presence of US troops on the island remains controversial, and there had been plans to build another base in Henoko, Okinawa. Prior to coming to power, Hatoyama had pledged Futenma would relocate, ideally abroad, but at least off the island.
A strong military presence in Japan has been the centrepiece of US strategy in East Asia. With the threat of Chinese military expansion, the Obama administration wanted Hatoyama to come up with an alternative to the complete withdrawal of US troops from Japan.
In the face of Washington's pressure and opposition from within, Hatoyama struggled to keep his pre-election pledge. In the end, Hatoyama resigned from his post, unable to deliver on his promises on the US military presence.
Hatoyama's inability to make a strong case for a credible alternative to the relocation made him "vulnerable" - not just to US pressure but from within the bureaucracy as well. In the end, this left his "regional vision in shambles and Japan all the more dependent on the US", said Selden.
Back to square one?
Hatoyama did not last a year in power. DPJ prime ministers since his resignation, Naoto Kan and Yoshihiko Noda, have dealt with smaller mandates and have moved closer to LDP policies. The measures taken by Hatoyama to limit the bureaucracy were abolished to help implement policy and the institution has since regained its power.
So what now for Japan? Will the LDP under the leadership of Shinzo Abe address the country's major structural problems?
"Japanese politics will continue to drift. And Japan's political leaders will do little more than follow its meandering course."
-Gerald Curtis, Columbia University
According to Curtis, Japan's governance problems will likely continue to play out short of "a crisis to shake the political elite out of its torpor - a military clash with China over disputed territory, for example, a provocative North Korean action against Japan, or a crisis in the government bond market".
Japanese people have a relatively high standard of living with low unemployment and "strong social bonds", wrote Curtis. "There is no evidence of a crisis mentality in Japan, even though politicians like to invoke that word."
Because ordinary Japanese have low political expectations of their leaders, they also do not rely on them and instead "do things for themselves".
"Japanese politics will continue to drift. And Japan's political leaders will do little more than follow its meandering course," Curtis predicted.