The DPJ is expected to win around 308 seats in the 480-member lower house of parliament.
"The people are angry with politics now and the ruling coalition," Yukio Hatoyama, the DPJ leader and the man who stands to be Japan's next prime minister, said.
He thanked voters, saying the election marked an end to "old politics and old government".
"Up to now it has been the bureaucrats who led the country," he said, referring the DPJ's election commitment to curb the powers of Japan's bureaucracy.
"From now on, I feel it is important that the people make decisions and we have to listen to the people."
Hatoyama said that he planned to form a coalition with some of Japan's smaller parties - the Social Democratic Party and the People's New Party.
The opposition already controls the less powerful upper house of parliament following elections in 2007, but its election win is historic as the LDP has ruled for all but 10 months since it was founded in 1955.
Earlier Hiroyuki Hosoda, the LDP secretary-general, said that he and other officials in the former ruling party planned to step down from their posts over their party's apparent defeat.
The LDP's defeat is being seen as a demand for change by voters disappointed with the LDP's performance and worried about a record jobless rate and a rapidly ageing society that is inflating social security costs.
Al Jazeera's Steve Chao, reporting from Tokyo, said: "Many people would say this has been a long time coming and the LDP should have seen the need for change."
In the build up to the elections, Hatoyama promised voters "revolutionary change", pledging to focus spending on social-welfare, including cash handouts for job seekers in training and families with children.
|Hatoyama promised voters an
end to 'old politics' [AFP]
But the Democrats will face an uphill struggle, with economic experts suggesting that their spending plans could inflate Japan's massive public debt and push up government bond yields.
The ruling party's loss will end a three-way partnership between the LDP, big business and bureaucrats that turned Japan into an economic juggernaut in the aftermath of the second world war.
That strategy foundered when Japan's "bubble" economy burst in the late 1980s and growth has stagnated since.
"This is about the end of the post-war political system in Japan," Gerry Curtis, a Japanese expert at Columbia University, said.
"It marks the end of one long era, and the beginning of another one about which there is a lot of uncertainty."