"So it gives the appearance that a group has been identified rather than individuals," he added.
France launched a country-wide crackdown on Roma earlier this month, a move that has been widely criticised by politicians and human rights groups.
The European Union is now reviewing if the move is legal and the Vatican has also spoken out against it.
Members of the UN panel also said they were concerned about "political speeches of a discriminatory nature in France".
These statements were made alongside "a recent increase in the act and manifestation of a racist and xenophobic characteristic," they said.
The French government said it sent back 283 Roma on Thursday, bringing the total number of Romanian and Bulgarian Roma deported so far this year to 8,313, against 7,875 expelled throughout last year.
The panel issued the recommendations two weeks after a two-day examination on whether France had applied the 1965 international convention on the elimination of all forms of racial discrimination.
During the hearing, the French delegation announced that a national plan to combat racism was being drawn up, without giving details.