Ehud Olmert's comments came as he addressed the House of Representatives and the senate during his visit to Washington on Wednesday.
Olmert urged other nations to curb Iran's nuclear programme.
"If we don't take Iran's bellicose rhetoric seriously now, we will be forced to take its nuclear aggressions seriously later," he said.
"The international community will be judged by its ability to convince nations and peoples to turn their backs on hatred and zealotry."
He said that Israel was committed to the US-sponsored "road map" peace plan, which calls for a negotiated settlement with the Palestinians - but he said that Israel will not wait forever.
"Our deepest wish is to build a better future for our regions, hand in hand with a Palestinian partner," he said. "But if they refuse, we will not give a terrorist regime a veto over progress, or allow it to take hope hostage."
He received a standing ovation when he said: "We will not yield to terror."
This was a reference to suicide attacks on Israelis such as those that killed a 16-year-old American in Israel this year. The boy's parents were in the audience.
Also on Tuesday, the White House signalled a policy shift by backing Israel's plan to set its borders unilaterally should Hamas refuse to disarm and recognise Israel.
George Bush, the US president, praised what he called Olmert's "bold ideas" for acting on his own in the event that talks on the "road map" peace plan collapse.