The opposition Democrat Party, which has 164 seats, and even some partners of Samak's own People's Power Party (PPP) stayed away from Friday's session amid strong dissent over his decision to re-contest the post.
On Thursday leaders of the PPP, the largest party in parliament, had endorsed him to return to office.
But other members of the ruling coalition criticised the move, casting doubt on whether Samak is the right man to lead Thailand out of weeks of escalating political turmoil.
They say his return to office would likely trigger further protests from the People's Alliance for Democracy (PAD), whose members have been occupying the prime ministerial office complex in Bangkok for more than two weeks.
"We honour the core party's nomination, but we think the new prime minister should be someone who can help resolve the political crisis," Somsak Prisana-anantakul of the Chart Thai Party, the coalition's second largest member, said.
Meanwhile unconfirmed reports suggested that more than a dozen legislators from Samak's party, claiming to speak on behalf of up to 70 other colleagues, would not vote for him.
A split in the ruling coalition raises the possibility of Abhisit Vejjajiva, leader of the Democrat Party, becoming a viable candidate for the top post.
Abhisit has called for a new prime minister to lead a non-partisan government of national unity made up of all parties and bring what he says is much needed stability to Thai politics.
"It doesn't have to include me or the Democrats," he told Al Jazeera. "I just want to see the new government reconciling and taking Thailand to its full potential."
But Giles Ungpakorn, a Bangkok-based political analyst, said such a move would be "a step backwards for democracy".
"Such a prime minister will not represent the majority of the electorate," he told Al Jazeera.
Friday's vote had been seen as a potentially critical juncture after weeks of instability and occasionally violent clashes between government supporters and protesters from the PAD.
With the threat of protests from both Samak opponents and supporters, hundreds of police had been deployed around the Thai parliament, equipped with batons and riot gear.