A spokesman for the European Union's foreign affair's chief described Gaddafi's call for jihad as "unforunate".
"If these reports are correct ... [they] come at an unfortunate moment when the European Union is working closely with Switzerland trying to reach a diplomatic solution" to a dispute between the two countries, Lutz Guellner, spokesman for Catherine Ashton, said.
Switzerland - with a population of 400,000 Muslims, mainly of Balkan and Turkish origin - has around 200 mosques, with just four minarets among them.
The ban drew condemnation from religious quarters around the world, including the Vatican.
"Any Muslim around the world who has dealings with Switzerland is an infidel [and is] against Islam, against Mohammed, against God, against the Koran"
Gaddafi, Libyan leader
Gaddafi said: "Any Muslim around the world who has dealings with Switzerland is an infidel [and is] against Islam, against Mohammed, against God, against the Koran."
"Let us fight against Switzerland, Zionism and foreign aggression," he said in a speech broadcast live on television.
"There is a big difference between terrorism and jihad which is a right to armed struggle."
Gaddafi also urged Muslims everywhere to boycott Swiss products and to bar Swiss planes and ships from the airports or seaports of Muslim nations.
The Swiss foreign ministry said it had no comment on Gaddafi's remarks.
Relations between Libya and Switzerland have been strained since July 2008 when Gaddafi's son Hannibal and his wife were arrested and briefly held in Geneva after two domestic workers complained he had mistreated them.
The row deepened when Libya swiftly detained and confiscated the passports of two Swiss businessmen, Rashid Hamdani and Max Goeldi.
Both men were convicted of overstaying their visas and of engaging in illegal business activities.
Hamdani's conviction was overturned in January, and he has now returned home, while Goeldi surrendered to authorities this week and is now serving a reduced jail term of four months.