The Paris Grand Mosque, World Islamic League and Union of French Islamic Organisations (UOIF) sued the magazine over its publication of two of the Danish caricatures and one of its own.
The Muslim groups said the cartoon showing a bomb in the Prophet's turban slandered all Muslims as terrorists, as did Charlie Hebdo's cartoon showing the Prophet reacting to Islamist fighters by saying: "It's hard to be loved by idiots."
"This is an attack on Muslims," Lhaj Thami Breze, UOIF president, had told the court. "It is as if the Prophet taught terrorism to Muslims, and so all Muslims are terrorists."
But the judges said that while the cartoon picturing the bomb in the Prophet's turban could offend Muslims if seen on its own, the picture had to be judged in the context of the magazine issue, which had discussed religious fundamentalism.
Even if the cartoon in itself was "shocking or hurting for Muslims, there is no deliberate desire to offend them", the court said.
Philippe Val, Charlie Hebdo chief editor, has said he published the caricatures in February last year after the editor-in-chief of the Paris tabloid France Soir was fired for reprinting them.