Voting stations opened at 06:00 GMT and will close at 16:00 GMT.
The poll comes just two days after Germany's parliament approved about $28bn in emergency loans over three years to Greece as part of a EU-International Monetary Fund rescue package.
A YouGov poll on Saturday said that 21 per cent of state voters would change their vote due to the Greek bailout.
"The issue has electrified people as seldom before and is going to play a determining role" in the election, Klaus-Peter Schoeppner, head of the polling institute Emnid, said.
Merkel has made more than 15 visits to the region recently and made frequent media announcements this week on why the aid to Greece, that is opposed by most Germans, is needed.
Many Germans believe the Greek loan money should be used to ease fiscal tightening at home.
"The election is extremely important because it is the first vote after the federal election and the poll in this big German state is being seen as a test for her," Gerd Langguth, a political scientist at Bonn University and biographer of Merkel, said.
An ending of the CDU and FDP majority in the Bundersrat, the upper house, would cause delays in policy enactment approved after last September's elections, including tax cuts.
Polls show that the alliance will not get a majority in the vote, in which 13.5 million citizens are eligable cast ballots.
However, Langguth said that if the CDU-FDP majority was lost, as expected, "it would not be a catastrophe but it would certainly make things more complicated and difficult for the chancellor".
Nadim Baba, Al Jazeera's correspondent in Dusseldorf said: "This region is extremely important in national politics here.
"The voters not only elect the members of the regional parliment but by doing so decide who is sent to the upper house of parliament.
"So with things neck and neck here, Merkel and her allies could loose control of the region and the upper house of parliament making it very hard for her to push through any reform plan quickly.
"That's why there is so much international interest, because not only is it seen as a referendum on Greece, but also an indication on how successful this government might be in the next year or so in terms of cutting Germany's budget deficit and getting its own finances in order."