In a 12-page report released on Friday, one day after Israel said it would speed up the building of the barrier, Annan said that he recognised Israel's "right and duty" to protect its people.
"However that duty should not be carried out in a way that is in contradiction to international law," he said, saying that its construction was a blow to hopes for a two-state peace plan being pushed by the United Nations.
"When each party should be making good-faith confidence-building gestures, the barrier's construction in the West Bank cannot... be seen as anything but a deeply couterproductive act."
The Israeli cabinet last year approved the globally condemned "apartheid wall", which the Palestinians claim is intended to redetermine the borders of any independent state they obtain in the future.
They say it is also depriving them of fertile land crucial to their troubled economy, a claim backed up by Annan in his report.
"Completed sections of the barrier have had a serious impact on agriculture in what is considered the 'breadbasket' of the West Bank," Annan said.
Wall is destructing propert, land
and business, says Annan
He added that checkpoints to allow Palestinian access to land already cut off by the barrier were frequently closed, cutting them off from farmland, hospitals, clinics, schools and essential services.
"Such access cannot compensate for the incomes lost from the barrier's destruction of property, land and businesses," Annan said.
Violations of rights
"This raises concerns over violations of the rights of the Palestinians to work, health, education and an adequate standard of living."
The UN secretary general said that when completed the barrier will run as deep as 22km into West Bank land and cut off some 400,000 Palestinians.
Annan's report was ordered last month by the UN General
Assembly, which passed a resolution condemning the construction of the barrier and calling on Israel to stop building immediately.