The appointment of the country's president is usually straightforward and the objections to Wulff on Wednesday highlighted how the bailing out of debt-burdened Greece and a series of domestic austerity measures have hit Merkel's popularity.
"Perhaps one or two people wanted to send a political signal to the leadership" Wolfgang Bosbach, a delegate with Merkel's Christian Democrats, told Phoenix television.
"Great idea, wrong day."
'Vote of no-confidence'
Alan Fisher, Al Jazeera's correspondent in Berlin, said the voting showed the weaknesses in Merkel's coalition government, which has been unstable from the moment it formed.
"From day one there have been divisions and strife and public arguments. This just shows how divided it is," he said.
"In Germany the role of president is largely a ceremonial one, but this was seen as much more than that. This was seen as a referendum on Angel Merkel's leadership."
Business newspaper Handelsblatt dubbed the vote a "debacle" and a "first vote of no-confidence" in Merkel. Bild, the mass-circulation daily, called the result a "sensation" and a "massive slap for the ruling coalition".
Influential weekly Die Zeit called the result a "humiliation" for Merkel's coalition, while Tagesspiegel described the vote as a "putsch" against the chancellor on what it termed the "day of the long knives."
Gauck, the candidate of the opposition Social Democrats (SPD) and the Greens, was popular with the public and there had been speculation that a victory for him could precipitate a political crisis.
He received a lengthy standing ovation from his supporters when his vote was announced in the parliament.
Polls suggested Gauck would have won a popular vote.
The ballot followed the resignation of Horst Koehler as president on May 31. He stepped down after being criticised for an interview he gave on the German military's role abroad.
Wulff becomes the country's youngest-ever president at the age of 51.
Favourite to take over from Wulff in Lower Saxony is David McAllister, who is half Scottish.