Democrats, including presidential candidates Barack Obama and Hillary Clinton had opposed the bill, over concerns about the paramilitaries targeting trade unions in Colombia and potential job losses in the US.
Republicans were angered by the move to delay the agreement, which Bush had said would send a message as a rebuke to "dictators and demagogues" in Latin America.
"The House Democratic leadership has now slapped around a major US ally," Susan Schwab, the US's trade representative, said after the vote.
"This is the Democratic leadership's version of foreign policy."
Congress voted to stop a 90 day clock under which it was required to act, after Bush had formally requested US lawmakers vote on the deal, meaning a vote on the deal was now unlikely to take place this year.
The move suspended part of the Trade Act of 1974, that lays out "fast-track" negotiating powers which bar any amendments to trade deals negotiated by the president.
The Senate is prohibited from taking any action on the bill until it has passed the House of Representatives.
Pelosi said it possible Congress could pass the agreement in 2008 if the White House and Democrats could agree on new legislation to bolster the US economy.
But for now, "Mr President, you simply don't have the votes," Pelosi said.
"If we are to be successful in passing a trade agreement, we have to first tell the American people that we have a positive economic agenda that addresses their aspirations, addresses their concerns."
John Boehner, the House of Representatives minority leader and a Republican, described Pelosi's decision to delay action on the trade pact was "nothing short of political blackmail."
"This vote today is a vote to kill the Colombia free trade agreement. Nothing more, nothing less," he said.
However, Change to Win, a US trade union movement representing seven million workers, welcomed the move, after campaigning against the deal due to the murders of trade unionists in Colombia.
"Colombia needs years, not months, to ensure the eradication of union killings and impunity that have plagued that country for decades," the group said in a statement.
On Monday, Bush had praised Alvaro Uribe, Colombia's conservative president, for his efforts to address concerns raised by Democrats by taking steps to reduce violence against trade unions and demobilise right-wing paramilitary fighters.