But allowing Iran to develop a nuclear weapon was also unacceptable, he said.
"Quite frankly, I am extremely concerned about both of those outcomes."
Asked if the military has a plan to strike Iran, Mullen replied "we do", without elaborating.
Iran insists its uranium enrichment programme is for peaceful purposes only, but Western powers believe it is covertly trying to develop nuclear weapons.
Responding to Mullen's comments, the deputy head of Iran's Revolutionary Guards said the strategic Gulf region will be unsafe for all if Iran comes under attack by the US.
"If the Americans make the slightest mistake, the security of the region will be endangered," Yadollah Javani told the official IRNA news agency.
"Security in the Persian Gulf should be for all or none. The Persian Gulf is a strategic region and if it is endangered, they [Americans] will suffer losses and our response will be firm.
"We will defend ourselves if America or Israel resort to any hostile measures against our vital values."
Mullen said he hoped a combination of international diplomatic efforts and sanctions against Iran would stop the alleged weapons development.
Last week, the state department said the US was prepared to follow up on a plan to provide fuel for Iran's research reactorin exchange for low-enriched uranium.
The plan was mooted in Geneva last October at a meeting between Iranian officials and senior diplomats from the five permanent UN Security Council members - Britain, China, France, Russia and the United States - along with Germany.