Slobodan Homen, a Serbian justice ministry official, said the decision to release Ceku from detention was a "political one".
"Once again, politics overcome international law," Homen told the state-run Tanjug news agency.
He said that Ceku would be "sooner or later available to Serbia's justice" and that "our ministry is still awaiting official information from our Bulgarian counterparts and an explanation why Ceku has been released".
Ceku was detained on an Interpol arrest warrant at the Gyueshevo border checkpoint near Kyustendil on Tuesday night, while he was entering Bulgaria from Macedonia.
Serbia has since officially requested his extradition, while Kosovo insisted that Ceku be released.
Belgrade's claim to have Ceku extradited was backed by Amnesty International, the UK-based human rights organisation, which "called upon the Bulgarian authorities to extradite promptly Agim Ceku to Serbia ... to face trial on war crimes charges".
The former premier and one-time military chief of Kosovo's Liberation Army of ethnic-Albanian fighters had been brought to the court on Thursday, heavily guarded and in handcuffs.
|Ceku was formerly military chief of Kosovo's Liberation Army [AFP]
Clad in an offical black suit and a tie, he told the court that the International Criminal Tribunal for the Former Yugoslavia at The Hague had already investigated crimes in the region and identified the people responsible for them.
Addressing the court, he said: "No charges were raised against me. I am not a Serbian national and never was.
"Serbia cannot request the extradition of foreign nationals [other than] its own."
Bulgarian media reported that his lawyers had presented the court with a letter from Hashim Thaci, the Kosovo premier, confirming that Ceku is a member of Kosovo's diplomatic community.
They also tabled a written statement from the United Nations, saying that the detained person was under the jurisdiction of the UN's interim administration in Kosovo.
Ceku was prime minister of Kosovo in the two years leading up to the disputed territory's unilateral declaration of independence from Serbia in February 2008.
He is wanted by Serbia for war crimes allegedly committed in Kosovo during the 1998 to 1999 war with Serbia.
Belgrade says Ceku's fighters killed 669 Serbs and 18 non-Albanians during the fighting.
It also accuses him of separate charges over his time as a leader in the Croatian army during that former Yugoslav republic's 1991 to 1995 independence war.
The Serbian indictments were issued in 2002, leading to Ceku's detention in Colombia earlier this year and in Slovenia and Hungary in 2003 and 2004, respectively.
Serbia has rejected Kosovo's declaration as a breach of international law and has said it will never recognise an independent Kosovo.