The loss of a majority in the state - and as a consequence in the federal upper house - means Merkel will have to rely on opposition parties for support for her policy programme.
The CDU and FDP have ruled in North Rhine-Westphalia - Germany's most populous state - since 2005, when they won 44.8 per cent and 6.2 per cent of the vote respectively.
Sunday's election came just two days after Germany's parliament approved about $28bn in emergency loans over three years to Greece as part of a EU-International Monetary Fund rescue package.
A YouGov poll on Saturday had indicated that 21 per cent of state voters would change their vote due to the Greek bailout.
Al Jazeera's Nadim Baba, reporting from Dusseldorf, said: "It has become clear that [Greece] has been a factor.
"There has been discontent all along about Germany's contribution towards the bailout of Greece."
Baba said control of the regional parliament in North Rhine-Westphalia could go to the Social Democrats and the Green party.
"There're already negative signs coming out of Merkel's Free Democratic party allies," he said.
"They're not happy. They've actually done worse than they were predicted, and Chancellor Merkel's own Christian Democrats are down about 10 per cent on the last time around.
Klaus-Peter Schoeppner, head of the polling institute Emnid, said the bailout had "electrified people as seldom before".
Merkel has made more than 15 visits to the region recently and made frequent media announcements this week on why the aid to Greece, that is opposed by most Germans, was needed.
Many Germans believe the Greek loan money should be used to ease fiscal tightening at home.
"The results show deep anxiety of voters with their government," Barbara Riedmueller, a political scientist at Berlin's Free University, said.
Gerd Langguth, a political scientist at Bonn University and biographer of the Chancellor, said: "The election is extremely important because it is the first vote after the federal election and the poll ... is seen as a test for her [Merkel]."
However, Langguth added the CDU-FDP majority loss "would not be a catastrophe but it would certainly make things more complicated and difficult for the chancellor".