"The constitutional court voted eight to six that the elections were unconstitutional and voted nine to five to hold a new election," said Judge Ura Wangomklang on Monday.
An opposition boycott of the election left empty seats in parliament and Thaksin Shinawatra, the prime minister, was unable to form a new government.
Thai academics had brought the case to the constitutional court, arguing that the elections were held too quickly, giving opposition parties inadequate time to prepare.
Abhisit Vejjajiva, the leader of the opposition Democrat party, welcomed the court decision, saying it was ready to re-join the electoral process
The party, which had argued that the poll was not fair, said it would participate in a rerun after a rare intervention by King Bhumibol Adulyadej last month.
Thaksin saw the election as a referendum on his leadership after months of street protests in Bangkok calling for his resignation over allegations of corruption and abuse of power.
A woman votes at a polling station
in southern Yala province (file)
After the inconclusive election, Thaksin said he was taking a "political break" and handed some tasks to his deputy.
In the past week he returned to the public eye, leading a cabinet audience with the king on Friday and fuelling speculation that his stepping aside was cosmetic.