Sri Lanka is one of 16 countries with GSP status.
In 2008, the European Union was Sri Lanka's largest export market, accounting for 36 per cent of all exports.
Suspending the tariffs would mean EU buyers would have to pay more for Sri Lankan exports.
Globally recognised brands like Marks & Spencer, Tesco and Next could take their business elsewhere, such as China, India and Vietnam.
The move would hit Sri Lanka's textile industry hard and thousands of job cuts as a result.
Rajiva Wijesinha, secretary of the Sri Lankan disaster management and human rights ministry, told Al Jazeera his country had responded to some of the "specifics" raised by the EU.
He said Sri Lanka "refused to submit to what is called a general investigation. But any specific thing we have said we will look at and this we are doing".
He did not confirm whether a formal response to the EU report had been made.
Wijesinha also accused the EU of being dishonest in its dealings with Sri Lanka.
"I think we have a situation where the EU is under a lot of pressure. We know that there are diaspora pressures; it's just that they are so dishonest about it," he said.
"The Americans, for instance, were much more honest in telling us that there was a report on certain things that was mandated by congress. I wish there was more honesty about these things."