The unrest included protests that shut down Bangkok's two airports, weakening the country's economy.
Abhisit's coalition is expected to at least hold their majority in the vote, for which more than 30,000 police officers have been deployed, as well as army and navy personnel in some provinces.
Alcohol has also been banned for the day in regions where voting is taking place.
Ten people were arrested on Saturday evening after a disturbance outside a Democrat Party office in northern Lamphun province, General Wichian Potphosri, the deputy national police chief, said.
Pravit Rojanaphruk, a political analyst with The Nation, an independent English newspaper, told Al Jazeera: "Most of the seats are in the north and north eastern regions, which have so far been a stronghold of the pro-Thaksin Shinawatra opposition party.
"However, some defections within the former pro-Thaksin coalitions, which have defected to the government side, may spell a difference today.
"So it's quite a significant by-election we are witnessing."
However, Rojanaphruk said that despite the defections, Thailand's political polarisation and confrontation could continue: "There has been a spate anti-government protests by the pro-Thaksin camp.
"One senior member of the pro-Thaksin group told me on Friday that perhaps they will not do something as drastic as shutting down the airport as the anti-Thaksin protesters had done back in November.
"But surely there will be more protest in the coming weeks and months."
The past leadership was pushed out after being some of its members were convicted of vote rigging.
Three parties in the previous governing coalition were dissolved after the ruling and the 29 available seats have been vacated by politicians disqualified from office.
The PPP beat the Democrats in elections in December 2007 after a period of military rule that had ousted Shinawatra in a coup in 2006.