Meanwhile in Brussels, a spokesman for the European Union's foreign affairs chief described Gaddafi's call for jihad as "unfortunate".
"If these reports are correct ... [they] come at an unfortunate moment when the European Union is working closely with Switzerland trying to reach a diplomatic solution" to a dispute between the two countries, Lutz Guellner, spokesman for Catherine Ashton, said.
France also condemned the comments, saying "such statements are unacceptable".
"The differences between Libya and Switzerland should be resolved by negotiation and we support the European Union presidency's efforts in this direction," Bernard Valero, a French foreign ministry spokesman, said.
Gaddafi made the remarks at a ceremony to mark the birthday of the Prophet Muhammad in Libya's Benghazi square on Friday.
"It is against unbelieving and apostate Switzerland that jihad ought to be proclaimed by all means," he said, calling Switzerland an "infidel, obscene state which is destroying mosques".
His comments come nearly three months after Switzerland voted in a referendum to adopt a ban on the construction of minarets, the towers that are a signature part of mosques.
'Fight against Switzerland'
Switzerland - with a population of 400,000 Muslims, mainly of Balkan and Turkish origin - has around 200 mosques, with just four minarets among them.
The ban drew condemnation from religious quarters around the world, including the Vatican.
"Any Muslim around the world who has dealings with Switzerland is an infidel [and is] against Islam, against Mohammed, against God, against the Quran"
Muammar Gaddafi, Libyan leader
Gaddafi said: "Any Muslim around the world who has dealings with Switzerland is an infidel [and is] against Islam, against Mohammed, against God, against the Quran."
"Let us fight against Switzerland, Zionism and foreign aggression," he said in a speech broadcast live on television.
"There is a big difference between terrorism and jihad which is a right to armed struggle."
Gaddafi also urged Muslims everywhere to boycott Swiss products and to bar Swiss aircraft and ships from the airports or sea ports of Muslim nations.
The Swiss foreign ministry said it had no comment on Gaddafi's remarks.
Relations between Libya and Switzerland have been strained since July 2008 when Gaddafi's son Hannibal and his wife were arrested and briefly held in Geneva after two domestic workers complained he had mistreated them.
The row deepened when Libya swiftly detained and confiscated the passports of two Swiss businessmen, Rashid Hamdani and Max Goeldi.
Both men were convicted of overstaying their visas and of engaging in illegal business activities.
Hamdani's conviction was overturned in January, and he has now returned home, while Goeldi surrendered to authorities this week and is now serving a reduced jail term of four months.