"They are aligned with a policy of attacking countries that are building new economic models.
"We aren't going to tolerate any foreigner coming here to try to sully the dignity'' of Venezuela.
'Attacking the messenger'
The two men were in Venezuela to present a report on Venezuela's human rights situation which said Chavez had encouraged discrimination against political opponents, stacked the courts and reduced freedom of expression during 10 years in power.
Vivanco, the group's Americas director, said that Venezuela expelled him "to avoid dealing with the issues, and distract attention by attacking the messenger".
"What happened is a confirmation of exactly the points that we raised in the report, and it shows the lack of tolerance in the government of President Chavez to criticism of his record on any area,'' he said from Sao Paulo, Brazil, where he arrived early on Friday.
The New-York-based group said the two activists were handed a letter accusing them of "anti-state activities" before their mobile phones were seized and their requests to contact their emabssies denied.
Human Rights Watch said it accepts no government funds, directly or indirectly.
The ejections came a week after Venezuela told the US ambassador to leave the country.
Chavez was elected president in 1998 after a campaign promising to improve the situation for Venezuela's poor majority. He was re-elected in 2000 and in 2006, but the US says he is a threat to democracy in Latin America.