Holbrooke has insisted that no such deal existed.
Karadzic, who Serb authorities say was captured on July 21, said Holbrooke's offer was made in 1996 to "the statesmen and ministers who were my authorised representatives".
In return, Karadzic said he was expected to "completely disappear" from the public arena, and refrain from giving interviews or publishing any works.
"In a word, become invisible long enough for the Dayton agreement to be implemented in full," he said.
Holbrooke also allegedly promised that Karadzic's Serbian Democratic party would not be prohibited from taking part in elections.
Karadzic said Madeleine Albright, the former US secretary of state, had proposed to Biljana Plavsic, the president of the Bosnian Serb republic at the time, "that I get out of the way and go to Russia, Greece or Serbia and open a private clinic".
He said he tried to meet his end of the deal, but it later became apparent there were attempts to have him killed.
Karadzic said: "It is clear that, unable to fulfill the commitments he had undertaken on behalf of the USA, he (Holbrooke) switched to plan B - the liquidation of Radovan Karadzic.
"The agreement, which should have brought me peace and freedom, thus became a source of great danger to my life, and to the life and safety of my family and even my friends."
Karadzic said Holbrooke's desire that he should disappear was "fresher and stronger" than ever.
"I do not know how long the arm of Mr Holbrooke or Ms Albright is... or whether that arm can reach me here," he said.
Muhamed Sacirbey, a former Bosnian foreign minister and former ambassador to the UN, told Al Jazera that there is a strong possiblity that a deal did exist.
On the day the deal was made, I met Robert Frowick, a diplomat, and head of the OSCE mission overseeing Bosnian elections at the time. He told me that Karadzic had agreed to withdraw from politics," he said.
"I agreed it was good idea, considering we didn't want a war ciminal running for president of Bosnia. But I asked what was given to him in return ... there was moment of quiet ... and Frowick confimrmed that there was deal that Karadzic would not be arrested."
"I have no reason to be on Karadzic's side ... but unfortunately ... it is quite possible that Holbrooke, on behalf of the United States, made this deal with him," Sacirbey said.
Holbrooke, credited with the Dayton peace agreement that ended Bosnia's 1992-95 conflict, rejected Karadzic's claims in an interview with US broadcaster CNN on Thursday.
"In June of 1996 I went to Belgrade and negotiated an agreement with Milosevic and two of Karadzic's henchmen that he would quit as president of the Serb portion of Bosnia, and as head of his political party, immediately and disappear from public life," Holbrooke said.
"He [Karadzic] reluctantly signed that agreement without ever coming to the negotiations, but then in order to protect himself, he put out this false story."
Carl Bildt, Sweden's foreign minister, who helped negotiate an end to the Balkans conflict, also denied the allegations.
"Such a deal never existed. There has not been any attempt to drop the charges against Karadzic," he said on Friday.
|Carl Bildt helped mediate an end to the Balkans war in the 1990s [EPA]
"Both Holbrooke and I were actively involved [in the 1990s]... in an attempt to get Nato to intervene to have him arrested. This failed, but that is another story," Bildt said.
Karadzic said that after initial attempts to kill him, he had decided to present himself to the tribunal.
But he claimed he was misled when investigators from the tribunal, who came to his hometown of Pale to seize evidence, turned out to be from the office of the ICTY prosecutor.
He said they "fooled us and rummaged through our archives without legal protection. It then became clear to me that I could not come before the tribunal because the conditions for a fair trial would not be met."
He also detailed "irregularities" with his eventual capture and handover to authorities last month.
"Unknown civilians showed me a badge so quickly that I could not identify it, took me out of a public transport vehicle and held me in an unknown place for 74 hours," he said.
Karadzic has been indicted on 11 counts of war crimes in connection with the 1990s Bosnia conflict.
He has indicated he will defend himself at the trial.
Karadzic's lawyer said that he was seeking the return of a laptop containing his client's entire defence from Serbian authorities.