"We came to check claims that at this location lie the remains of an undetermined number of people," Milan Diplaric, the investigating judge, said.
Forensic experts from the Sarajevo-based International Commission for Missing Persons are also at the site to help identify any remains through DNA analysis.
"We are assisting this investigation with the participation of our forensic experts," Vedran Persic, ICMP spokesman, said.
"We're also ready to eventually identify [any victims] as we have a database with 84,000 [DNA] samples from families whose members went missing" during the Balkan wars of the 1990s, he added.
A source told the AFP news agency that the investigation was launched after an account by a protected witness to the International Criminal Tribunal for the former Yugoslavia.
"The location was provided by the Hague tribunal. An anonymous witness said that they saw trucks dumping the bodies," the source said.
A senior Serbian official told Reuters news agency that he believed the bodies were originally buried elsewhere, then dug up, loaded onto four trucks, and dumped at the quarry on June 3, 1999.
The grave would be the largest discovered in Serbia since 2001, when more than 800 dead Albanians were found in two sites in Serbia.
Kosovo has been under United Nations stewardship since June 1999 when a Nato assault drove out Serb forces waging a brutal crackdown on separatist ethnic Albanian rebels and their civilian supporters.
The UN Security Council will decide soon on a plan that would make Kosovo independent under international supervision.
According to the International Committee of the Red Cross, about 2,400 people, most of them ethnic Albanians, remain listed as missing almost eight years after the end of the conflict.