The South American nation hired the public relations firm to help win US congress approval for a free trade agreement with the US.
The issue has plagued the campaign of the New York senator, who is vying with Barack Obama to become the Democratic nominee to run against John McCain, the Republican candidate, in the November election.
A Clinton campaign source told Reuters that Clinton had been disappointed over Penn's meeting.
During the weekend, Penn decided he should step down, and "Senator Clinton agreed with him", the campaign source said.
The two Democratic candidates have sparred over trade, with each questioning the other's credibility in their pledges to renegotiate the North American Free Trade Agreement.
Free trade scepticism
Scepticism about free trade runs deep among the working-class voters Clinton and Obama are courting, and both Democratic candidates oppose the deal with Colombia.
Penn said last week he had made "an error in judgment" in meeting the Colombian ambassador.
The Colombian government retaliated by firing Burson-Marsteller, saying that it "considers this a lack of respect to Colombians, and finds this response unacceptable".
The president of the powerful Teamsters union, James Hoffa, said Penn's meeting with the Colombian officials undermined Clinton's stance on labour and trade issues.
Obama has said that Clinton has close ties to lobbyists who might wield undue influence shaping policies, should she become president.
The Clinton campaign said Penn and his polling firm Penn, Schoen and Berland Associates would continue to provide polling services and advice to the campaign.
Penn served as pollster and political adviser to Bill Clinton, the former US president and Hillary's husband, and also worked on Hillary's successful first campaign for the US senate.
She was elected in 2000 and re-elected in 2006.
Penn has been criticised for taking an aggressive approach to Clinton's presidential bid, including trying to make an issue of Obama's drug use as a young man, which the Illinois senator has admitted.
Labour leaders earlier criticised the Clinton campaign over Penn's dual roles, saying that Burson-Marsteller was involved in anti-union activities, including helping a corporate client resist union organising.