Some 175 million people, or 3% of the world's population, live outside the country where they were born, Annan said on Friday in a lecture at New York's Columbia University.
The $88 billion migrant workers send back to their home countries each year and the resulting cultural diversity should be cause for celebration.
But the phenomenon has also raised concerns that migrants pose a terrorist threat, are trying to take away peoples' jobs or could deplete limited social services budgets, Annan said.
"These are understandable concerns and they must be answered. The answers are not easy. But I have come here today to say that they do not lie in halting migration, a policy that is bound to fail," Annan said in prepared remarks. "I say the answer must lie in managing migration, rationally, creatively, compassionately and cooperatively."
Annan's speech, billed as a major policy address, comes
during a US crackdown on immigration as part of its global "war on terror" and amid years of European efforts to exclude foreigners to protect domestic jobs.
Countries accepting immigrants often fail to accept them as equals or grant them legal status, even as they benefit from
their labour, Annan said.
Migrants "are usually not free riders looking for an easy life but courageous people who make great sacrifices in search of a better future for themselves or their families," he said.
3% of world's population live
outside their country of origin
"Nor are their lives to be envied once they have left home. They often face as many risks and unknowns as they do hopes and opportunities. Many fall prey to smugglers and traffickers on their journey, and many more face a surly welcome of exploitation, discrimination and prejudice once they arrive."
While "homeland security" was vital, it would be a tragedy for the United States "to deprive itself of the enrichment of many students and workers and family members from particular parts of the world, or if the human rights of those who would migrate here were compromised," he said.