In an interview with the CBS news show "60 Minutes" that will air on Sunday, Pervez Musharraf said the threat came from Richard Armitage, the US deputy secretary of state, and was given to Musharraf's intelligence director.
Musharraf said: "The intelligence director told me that [Armitage] said, 'Be prepared to be bombed. Be prepared to go back to the Stone Age.'
"I think it was a very rude remark."
Armitage was not immediately available to comment. A Bush administration official said there would be no comment on a "reported conversation between Mr Armitage and a Pakistani official".
But the official said: "After 9/11, Pakistan made a strategic decision to join the war on terror and has since been a steadfast partner in that effort. Pakistan's commitment to this important endeavor has not wavered and our partnership has widened as a result."
Interest of the nation
Musharraf is now in Washington and is due to meet George Bush in the White House on Friday.
The Pakistani leader said he reacted to the threat in a responsible way.
"One has to think and take actions in the interest of the nation, and that's what I did."
Before the September 11, 2001 attacks, Pakistan was one of only a few countries to maintain relations with the Taliban and many Pakistanis were sympathetic with the neighbouring Islamic state.
But within days of the attacks Musharraf cut his government's ties to the Taliban regime and cooperated with US efforts to track and capture al-Qaeda and Taliban forces that sought refuge in Pakistan.
The official 9/11 Commission report on the attacks and their aftermath, based largely on government documents, said US national security officials focused immediately on securing Pakistani cooperation as they planned a response.
Documents showed Armitage met the Pakistani ambassador and the visiting head of Pakistan's military intelligence service in Washington on September 13 and asked Pakistan to take seven steps.
They included ending logistical support for bin Laden and giving the United States blanket overflight and landing rights for military and intelligence flights.
The report did not discuss any threat the United States may have made, but it said Musharraf agreed to all seven US requests the same day.
Lisa Curtis, a South Asia specialist with the Heritage Foundation think tank in Washington, said she did not know exactly what was said by Armitage but was skeptical he would have threatened to bomb Pakistan.
"The question of any bombing taking place, that question revolves around Afghanistan.
"I would find it difficult to believe he talked about bombing Pakistan specifically because, while I don't know the exact contents of the conversation, I do know it was a pretty firm ultimatum in terms of ... choosing between the Taliban or the US."
With the Taliban still fighting in Afghanistan and statements by the Afghan government that Pakistan must do more to crack down on pro-Taliban fighters in its rugged border area, the issue is again a sensitive one between Islamabad and Washington.
Musharraf reacted with displeasure to comments made by Bush on Wednesday that if he had firm intelligence bin Laden was in Pakistan, he would issue the order to go into that country.
Musharraf told a news conference: "We wouldn't like to allow that. We'd like to do that ourselves."