The Intermediate-Range Nuclear Forces Treaty was signed in 1987 by then-US President Ronald Reagan and Soviet General-Secretary Mikhail Gorbachev in Washington.
It banned nuclear and conventional missiles with ranges of 500 to 5,500km, as well as their launchers.
The two countries have long accused one another of violating the terms of the landmark treaty.
"Russia has not adhered to the agreement," Trump told reporters in Elko, Nevada, without giving any further details.
"We're going to terminate the treaty and we're going to pull out," he added.
When asked what that meant in practice, he said" "We'll have to develop those weapons."
Trump made the comments as his National Security Adviser John Bolton was in Moscow to meet Russia's Foreign Minister Sergey Lavrov, before what is expected to be the second summit between Trump and his Russian counterpart, Vladimir Putin, later this year.