Analysts said the results reflected a desire by voters for a two-party system to challenge the long-ruling Liberal Democratic Party (LDP). They also said Koizumi's personal popularity could be fading.
The three-party coalition led by Koizumi's LDP won 275 seats out of 480 in the lower house in Sunday's polls, according to official results from constituencies compiled by the media.
The LDP alone won 237 seats, down from 247, meaning the party lost the outright majority it previously held in the lower house and will now need the support of coalition partners to govern.
The largest opposition Democratic Party of Japan (DPJ) won an unexpected 177 seats, up from 137 before the election, raising the prospect of a genuine two-party system after half a century of near total LDP rule.
The full official results were due to be announced on Monday evening.
"The political landscape is about to suddenly turn into a two-party system," the top selling Yomiuri Shimbun newspaper said in its editorial.
"Even if Koizumi continues to bear the responsibility of setting the future course of this country, he is likely to continue to face difficulties in managing the government," it said.
The lower house election was the first chance for 103 million voters to pass judgement on the two and a half year-old Koizumi government, which preached reform to end more than a decade of economic stagnation.
"Since the LDP has ruled a long time, I suspect people are wanting change and the DPJ's strong gains reflect such a wish"
Japanese prime minister
Despite the apparent setback to his reform agenda, Koizumi was upbeat when he spoke to reporters on Monday, saying he felt "good" and saying he believed voters had supported his reforms.
But shortly after the polls closed Sunday, a chastened prime minister acknowledged that people were impatient for change after 48 years of virtually unbroken LDP rule.
"Since the LDP has ruled a long time, I suspect people are wanting change and the DPJ's strong gains reflect such a wish," Koizumi said.
"There are lots of people who are becoming impatient for the results of reforms."
Newspaper headlines on Monday suggested the reform process could now face a rocky ride and the effect of Koizumi's charm and charisma could be fading.
"Winds blow against his structural reforms," read one headline, "Koizumi magic fading," said another.
Koizumi took power from his deeply unpopular predecessor, Yoshiro Mori, two and a half years ago advocating drastic economic reforms and scored an easy win in upper-house elections in July 2001.
"(Koizumi) had nothing to boast of, except for some improving signs in the economy"
politics professor at Tokyo University
The deeply conservative LDP old guard which is opposed to his policies, have kept the maverick politician in power as a vote-collecting machine for elections while effectively reducing his reform drive to empty rhetoric.
Analysts said the prime minister's failure to deliver an absolute LDP majority would not spell an end for him, but they warned it would make it more difficult for him to push through some of his policies.
"Mr Koizumi had loudly pitched for 'structural reforms' but voters have not seen any major change," said Takashi Inoguchi, politics professor at Tokyo University.
"He had nothing to boast of, except for some improving signs in the economy," Inoguchi said.
"Dark clouds are hanging over the LDP and Mr Koizumi ... If things carry on like this, the DPJ will make another leap in upper-house elections next year, shaking the LDP," he said.
During the election the pension system - strained by the rapid greying of Japan's population - emerged as a key issue while Koizumi played down his unpopular plans to send troops to Iraq.
The liberal-leaning DPJ vowed to fight for those excluded by what it called a political structure of vested interests. It also vowed to trim the bureaucracy, cut wasteful spending and boost jobs.