Annan said on Friday that both sides had to "lower the rhetoric" and intensify diplomatic efforts to find a solution satisfactory for all sides.
"I have insisted very clearly ... that I think it's important that the United States comes to the table, and that they join the European countries and Iran to find a solution.
"I urge all parties to be open, Iran included, and come back to the table."
While Washington has said it favours a diplomatic end to the dispute, it has not ruled out military force and has been pushing for a tough UN Security Council resolution against Iran which could include sanctions.
However, several European countries are working on a plan which will present Iran with a choice of incentives or sanctions.
On Thursday Mahmoud Ahmadinejad, the Iranian president, said during a visit to Indonesia that he was ready to negotiate about his country's nuclear programme, but said that any threats made against his country would make negotiations difficult.
Iran says that its nuclear programme is only for peaceful purposes, but Western countries suspect it is to develop nuclear weapons.