It was right to condemn attacks such as those by al-Qaida on the United States on 11 September, 2001, but people must beware of polarising the West and Islam, Annan said on Friday, according to the text of a lecture to Tuebingen University in southern Germany.
It is wrong to behave "as if Islamic and Western values were incompatible," he said. "They are not, as millions of devout Muslims living here in Germany, and elsewhere in the West, would be the first to tell you."
"Yet many of those Muslims now find themselves the objects of suspicion, harassment and discrimination, while in parts of the Islamic world anyone associated with the West or Western values is exposed to hostility and even violence," he said.
Last week, Annan urged the United States to take a more multilateral approach to combating the threat of armed extremists and to be more patient in building alliances if it wanted more support for activities in Iraq and elsewhere.
"Misunderstandings" have arisen between the Islamic world and the West and amid such acrimony the relevance and importance of the United Nations have been called into question, he said.
By targeting Islamist groups around the world, US President George Bush's war on terror has created tension with some Muslim countries despite assertions by Bush and his allies that the war is not directed against the religion of Islam.
Since September 11 2001, attacks on Muslim targets have included arson at an Islamic school in the Netherlands, assaults on Afghan men in Britain and the removal of Muslim women's veils in Belgium, and Muslims across Europe have said they feel unsafe.
Annan says Muslim and Western values are not at odds
Many of the United States' seven million Muslims said they felt discrimination against them increased after the September 11 attacks. A poll taken after September 2001 showed three quarters of US Muslims had experienced some form of bias or knew a Muslim who had.
A British survey last year showed that 69% of the country's 1.8 million Muslims felt excluded from mainstream life, while almost two-thirds said relations with non-Muslims had deteriorated since the September 11 attacks.
Members of Germany's three-million strong Muslim community said they felt increased suspicion of them, their schools and mosques after it emerged that Germany had been a haven for the al-Qaida plotters, several of whom had lived in Hamburg for years.
A major survey published this week showed that more than one-quarter of Germans thought Muslims should not be allowed to move to Germany.
Optimism over Middle East
On Thursday Annan said he could detect a new mood for peace in the Middle East which he hoped would inspire Israeli and Palestinian leaders to return to the negotiating table.
Speaking on a visit to Berlin, he admitted the so-called roadmap for peace was "in a lot of distress".
Annan (L) and Schroeder both back a peace plan 'in distress'
"Distress does not mean dead. We hope we can bring it back," he added.
Annan told a news conference after talks with Chancellor Gerhard Schroeder that there was "a new mood in the region", helped in part by the signing last week of an unofficial alternative peace plan.
That plan, known as the Geneva initiative, makes proposals to resolve some of the thorniest problems in the Palestinian-Israeli conflict. It has been widely criticised by both sides.
"This new dynamic, I hope, will inspire the leaders of both parties to take the necessary steps to get back to the negotiating table," Annan said.