"They took this up and essentially they created a situation in which it became apparent that my presence at the National Intelligence Council would be constantly used to denigrate its products and to discredit them.
"I concluded with Admiral Blair [Dennis Blair, director of National Intelligence] that it would be in the best interests of the council and my country for me to step down."
Benyamin Netanyahu, Israel's prime minister-designate and leader of the Likud party, is attempting to form government that could see Avigdor Lieberman, the leader of Yisrael Beitenu, become foreign minister.
Freeman also told Al Jazeera that members of US congress had a "basic intolerance ... for any viewpoint that didn't conform to their preconceptions or policy preferences".
Freeman, currently president of the Middle East Policy Council think-tank in Washington DC, also said the controversy over his nomination raised concerns that intelligence could be "sliced and diced" to suit people's agendas.
"Is intelligence simply ammunition for polemic and arguments on behalf of policies that are already decided?" he said.
"The whole incident raises a serious question about the extent to which our intelligence community will be allowed to be objective."
However, he said the fact his withdrawal had sparked such public interest had "eroded a taboo" for public discussion on how much power groups such as the American Israel Public Affairs Committee (Aipac) wielded.
"I hope that our media and public debate can now open up and allow criticism of Israeli policies that people like myself regard as deeply injurious to Israeli as well as to our own country's public interests ... That has not been the case," he said.
Members of pro-Israel groups in the US and both Republican and Democratic politicians raised concerns over Freeman's nomination, citing comments the former ambassador had made over "the brutal oppression of the Palestinians by Israeli occupation" which they said showed bias.
Freeman countered shortly after he withdrew his nomination that the lobby had plumbed "the depths of dishonour and indecency" in a "barrage of libellous distortions" regarding his public record.