In recent weeks, the US has imposed new sanctions on Iranian nationals and entities, accusing it of breaching the nuclear deal, allegations that Iran denies.
"Today is the time for the mother of all negotiations, not the mother of all bombs," Rouhani said on Saturday as he addressed parliament members and foreign guests in Tehran.
Among the guests for Rouhani's inauguration on Saturday was Federica Mogherini, the EU foreign policy chief, who has pledged to continue building ties with Iran despite mounting pressure from the US to keep the country isolated.
Meeting Mogherini before the ceremony in parliament, Rouhani called for greater efforts to uphold the 2015 nuclear deal struck by Iran and world powers.
"Repeated breaches of commitments by the US administration ... can be destructive," he said, quoted by the official IRNA news agency.
"All sides have a heavy responsibility to safeguard this agreement which was achieved by a lot of effort."
Rouhani vowed a "unified" response to any breach of the landmark agreement.
"The world should know that any breach of the deal will face a unified reaction of the Iranian nation and government," he said.
Some of Iran's older friends, including Zimbabwean President Robert Mugabe, also arrived for the ceremony, which came two days after Rouhani was officially sworn in by the country's supreme leader Ayatollah Ali Khamenei.
The emir of Qatar, Sheikh Tamim bin Hamad Al Thani, was absent despite attending Rouhani's last inauguration in 2013.
Among Iranians, the real attention was on who would be named to Rouhani's new cabinet.
He has already been facing criticism over indications that women would again be entirely absent and that his reformist allies would barely be represented.
Rouhani's last government had three women among a large cohort of vice presidents, but they lacked ministerial roles that would require approval by parliament.
"It was the reformists that allowed him to win the elections in 2013 and 2017... he must listen to those who supported him," Rassoul Montajabnia, of the reformist National Confidence party, told the Arman newspaper.
Rouhani won a convincing victory over Ebrahim Raisi in the May presidential election, promising to continue rebuilding ties with the West and easing social restrictions at home.
That helped win over the reformists - whose candidates dropped out of the last two elections to ensure his victory.
"Rouhani created a lot of expectation and now there is a sense that he is retreating from his promises," said Ali Shakourirad, head of the reformist People's Unity Party.
The absence of women was down to pressure from religious conservatives behind the scenes, Shakourirad said.
"Rouhani didn't want to make his task any more complicated than it already is," he told AFP news agency.
The final line-up is due to be announced in coming days, and must then be approved by parliament.
Despite the furore, the reshuffle is not expected to touch big names such as Mohammad Javad Zarif, foreign minister, and Bijan Namadar Zanganeh, the oil minister.
"The main ministers are very likely to stay in place," said Henry Smith, Iran analyst with consultancy Control Risks, told AFP.
Speaking to Al Jazeera, Arron Reza Merat, a London-based Iran analyst, said that while Rouhani's victory may be a reflection of the increasing influence of reformists, second-term presidents in Iran "do turn into lame ducks", and that the supreme leader and the security establishment have "larger sway" in the country.