The UNHCR has registered more than 9,000 refugees in the country, mostly would-be migrants who tried to enter Italy from the Middle East and North Africa. The Italian government expels them to Libya.
"This will leave a huge vacuum for the thousands of refugees and asylum seekers who are there already, and of course those who continue to arrive steadily on boats every week," Fleming said.
Palestinians are the largest group of refugees in Libya, followed by people from Iraq, Sudan, Somalia and several other countries. Most live in cities, though some are confined to detention centres.
The UNHCR provides health care, shelter and education for refugees.
"All European governments considering using Libya as a place where people fleeing from war and persecution could be received would have to review this very carefully if UNHCR is no longer present there," Fleming said.