"I don't think that the whole army supported this interruption of the democratic system by capturing a president elected by the people.
"I think that this has been a plot by an elite whose only wish is to keep the country isolated and in total poverty."
Congress named Roberto Micheletti, the current speaker, as the country's new president.
Earlier, Zelaya's supporters gathered outside the presidential palace, shouting insults at the soldiers inside and setting fires in the street, after news of his arrest emerged.
"They kidnapped him like cowards," Melissa Gaitan, an employee of the official government television station, said, referring to Zelaya.
|Zelaya was elected for a non-renewable
four-year term in 2006 [File: AFP]
"We have to rally the people to defend our president."
Al Jazeera's Mariana Sanchez, reporting from Tegucigalpa, said that people had set up barricades around the building.
"A lot of people are wielding sticks and steel batons and they are very angry. At one point they tried to push their way into the gates of the palace, but the army inside resisted," she said.
"There are some people among the protesters who are trying to calm people down.
"They have come with loudspeakers and they are telling people that they are too few to go into the presidential palace."
Many union, labour and farm movements support the non-binding referendum, which Zelaya says is aimed at improving the lives for the nearly three-quarters of Hondurans who live in poverty.
Oscar Hendrix, a youth movement leader in the northwestern city of San Pedro Sula, said that the military had burned the ballot papers that had been distributed in defiance of the supreme court ruling.
"This is inconceivable. This is one of the fundamental rights of the people," he told Al Jazeera.
Hendrix said that there would be protests against the military's actions.
"We are analysing right now whether we are going to do something here or whether we are all going to mobilise to the capital city," he said. "We will stand up for our rights."
The referendum, which was due to take place on Sunday, would have asked Hondurans whether they approved of holding a poll on constitutional change alongside general elections in November.
The supreme court, which last week ruled that the vote could not go ahead because the constitution bars changes to some of its clauses, such as the ban on a president serving more than one term, said it ordered the military to remove Zelaya.
"Today's events originate from a court order by a competent judge," it said in a statement.
"The armed forces ... acted to defend the state of law and have been forced to apply legal dispositions against those who have expressed themselves publicly and acted against the dispositions of the basic law."
Mark Weisbrot, co-director of the Washington-based Centre for Economic and Policy Research, said that it seemed the military, with the backing of the court, had used the referendum as a "pretext" to overthrow the government.
"This was a fight over a non-binding referendum, nothing more than a poll of public opinion, so no one can really make the argument that there was some kind of irreparable harm that would take place if the president got his way," Weisbrot said.
Zelaya was elected for a non-renewable four-year term in 2006 as a member of one of Honduras's established conservative political parties.
However, since taking power Zelaya has moved to the left, aligning himself with Hugo Chavez, Venezuela's president.
Colin Harding, an expert in Latin American politics, told Al Jazeera that Zelaya had apparently overestimated his own power in pushing for the referendum.
"He has no support in within his own party, he is opposed by congress, he is opposed by the judiciary and the military, who are not the power they used to be but have lined up against Zelaya ostensibily in defence of legality," he said.
Chavez has threatened military action in Honduras if Patricia Rodas, Venezuela's ambassador in Tegucigalpa, is harmed. He said that she had been abducted by soldiers and beaten earlier in the day.
"This military junta that is now there would be entering a de facto state of war," he said.