Condoleezza Rice made the announcement at the State Department on Monday.
Zoellick, the deputy secretary of state, who served six years in the Bush administration, said he would join the Wall Street investment house Goldman Sachs Group.
Rice praised Zoellick's "tireless work ethic" and said he had served as her "alter ego" in the department.
"Our nation is stronger and safer because of your work," she said.
In his resignation letter, dated June 15, Zoellick, 52, did not say why he was leaving.
"I'm pleased to have built a first-rate team, offer counsel on a number of initiatives and backup the secretary," Zoellick said at the news briefing.
"I've accomplished what I set out to do, and it's time to step down."
Zoellick is expected to leave the State Department in July, officials said.
For several months, Zoellick has been speaking to Wall Street firms about moving to a more lucrative position there.
Before joining the State Department in February 2005, Zoellick served as US trade representative where he completed negotiations to bring China and Taiwan into the World Trade Organization.
Several sources said recently it had been hard for Zoellick to accept his second-term appointment to the post, even though it was for a State Department job where he had considerable independence.
Last month, he played a pivotal role in getting the main rebel groups in Sudan's western Darfur region to sign a peace agreement after talks had dragged on for years in the Nigerian capital Abuja.