European Council President Donald Tusk and European Commission President Jean-Claude Juncker signed the agreement, together with Japan’s Prime Minister Shinzo Abe, at a ceremony in Tokyo.
The pact creates the world’s largest open economic area amid fears that a trade war between the US and China will diminish the role of free trade in the global economic order.
“There are rising concerns about protectionism, but I want Japan and the EU to lead the world by bearing the flag of free trade,” Abe said at a news conference after the signing ceremony.
“We are sending a clear message that we stand against protectionism. The EU and Japan remain open for cooperation,” Tusk, who speaks for the 28 EU national leaders, told reporters.
Juncker said the signatories of the agreement were making “a statement about free and fair trade, we are showing that we are stronger and better off when we work together”.
The deal now requires ratification by the European Parliament – a move expected in the coming months – and by Japanese legislators.
The deal, which is the largest ever negotiated by the EU, will unite about 600 million people across both markets. EU officials hope it will come into effect by early 2019.
Under the pact, Japan will remove tariffs on 94 percent of all imports from the EU, including 82 percent of farm and fishery products.
The EU will eliminate tariffs on 99 percent of imports from Japan.
It will abolish tariffs on Japanese cars and trucks in the eighth year and televisions in the sixth year following the implementation of the pact.
The signing had been due to take place in Brussels on July 11, but Abe was forced to cancel his trip after torrential rains triggered flooding and landslides in Japan that killed about 220 people.
The US this month imposed 25 percent tariffs on $34bn of Chinese goods to lower the US trade deficit, and China quickly retaliated with an increase in tariffs on US goods.
The Japan-EU trade deal is also a sign of shifting global ties as Trump distances the US from longtime allies like the EU, NATO and Canada.