The Federal Administrative Court said a law passed in April by the south-western state of Baden-Wuerttemberg was unfair, according to the news magazine Der Spiegel on Monday.
Judges said the ban was only being applied to Muslim women.
Nuns, who often work in public schools in the predominantly Roman Catholic Black Forest region of the state, will now be required to remove their habits before entering the classroom.
"Exceptions for certain forms of religiously-motivated clothing in certain regions are out of the question," the federal judges of the Leipzig-based court said.
The author of the state legislation, law professor Ferdinand Kirchhof, told Der Spiegel that nuns' habits were considered to be "professional uniforms" in the region and thus exempt from the religious symbols law.
Germany's highest tribunal, the constitutional court, ruled in September 2003 that Baden-Wuerttemberg was wrong to forbid a Muslim teacher from wearing a headscarf in the classroom.
But it said Germany's 16 states could legislate independently to ban religious apparel if it was deemed to unduly influence children. This has subsequently created a patchwork of varying rules throughout the country.
"Exceptions for certain forms of religiously-motivated clothing in certain regions are out of the question"
German federal judge ruling
Muslim groups have fiercely criticised the ban for compromising their freedom of religious expression. Germany is home to more than three million Muslims, making Islam the country's third largest religion.
German state laws stop short of limits set by controversial new legislation in neighbouring France, which outlaws both teachers and students from donning all conspicuous religious insignia in state schools.