"Secretary Powell is undergoing surgery this morning for prostate cancer," the State Department said on Mondy, adding, the operation was being performed at Walter Reed Army Medical Center in Washington.
His department confirmed Powell would be in hospital for "several" days, and his workload would be reduced to allow him to recover.
Adam Ereli, spokesman for the State Department, described the operation as "routine intervention", and said it had been scheduled for some time. President George Bush is said to have been notified about the operation two weeks ago.
State Department officials, including Powell's own aides, only learned of the operation at a staff meeting on Monday morning, reports say.
Deputy Secretary of State Richard Armitage was in charge of the State Department in Powell's absence, a White House spokesperson confirmed.
There has been already much speculation Powell will leave the administration if President Bush wins a second term in office next year. Today's news can only increase the chances of Powell announcing his retirement from politics.
The 66-year-old Powell has been a familiar figure on the political stage since 2001. He has a distinguished military background and rose to the American public's attention during the Gulf war in 1991.
He is not the only member of the Bush administration to have suffered ill health. Vice President Dick Cheney was in hospital in 2000 with chest pains, suffering a suspected heart attack.
Cheney, 59, suffered three heart attacks more than a decade ago whilst campaigning for George Bush to become US president.