Jean-Claude Juncker said on Wednesday "the Americans can't just get out of the agreement," adding "it takes three to four years" to pull out.
Juncker went on to say the Group of Seven leaders "tried to explain this in clear, simple sentences to Mr Trump" at a recent summit in Italy. He said even though "it looks like that attempt failed", the "law is the law".
In a jibe at the US administration, Juncker told the audience at an event of the Confederation of German Employers in Berlin "not everything that is written in international agreements is fake news".
"If the US president pulls out of the Paris agreement, and he will in the next days or hours, then it is Europe's duty to say that that is not how it works," Juncker said.
A White House official said earlier in the day that Trump was planning to pull out of the Paris deal, although a final decision hadn't been made.
Trump on Wednesday declared abandoning the Paris Agreement would be a victory for the American economy.
The European Union and China, meanwhile, will reaffirm their commitment to the Paris Agreement this week, regardless of whether the US pulls out of the pact, a senior EU official said.
The official told reporters that the EU and China would also "spell out" how they plan to meet their commitments to the landmark international accord to fight global warming at talks in Brussels on Friday.
The official is involved in preparing the meeting between EU Council President Donald Tusk, Juncker, and Chinese Premier Li Keqiang, but can't speak on the record because the meeting statement wasn't finalised.
Li and a major Chinese delegation are due to arrive in Brussels late Thursday following talks in Berlin.
"The EU and China are joining forces to forge ahead on the implementation of the Paris Agreement and accelerate the global transition to clean energy," EU climate commissioner Miguel Arias Canete said about the upcoming EU-China summit.
A White House official said Wednesday there could be "caveats in the language" announcing a withdrawal, leaving open the possibility that this decision isn't final.
That possibility was met with derisive howls from EU lawmakers when a session of the European Parliament was informed about it.
"Climate change is not a fairy tale. It is a tough reality which affects peoples' daily lives," European Parliament President Antonio Tajani said in a statement. "People die or are obliged to leave their homes because of desertification, lack of water, exposure to disease, extreme weather conditions. If we don't act swiftly and boldly, the huge human and economic cost will continue to increase."
Tajani suggested Washington's withdrawal should be a signal for Europe to step up its efforts - and reap the benefits.
"Our climate action strategy represents an opportunity to attract investment, innovation and develop new green technologies," he said. "We have got the talent and the will to make this possible in all sectors."
Tajani said earlier he would confer with Tusk and Juncker about "joint initiatives to be adopted together as a European Union" to offset the decision.
The EU official involved in organising the EU-China meeting said it would "send important signals for the multinational system", as Trump moves to unpick some of the international trade agreements the US has signed on to.
"I don't think any other countries will follow the US out of Paris, so if he does leave, Trump will be in splendid isolation with the leaders of Syria and Nicaragua," said Alden Meyer, director of strategy and policy for the Union of Concerned Scientists advocacy group.
Source: AP news agency