Speaking to reporters in Bangkok on Friday, Annan said Israel is responsible for complying with the decision.
"I think the decision of the court is clear," he told reporters in Bangkok, where he is due to attend a world AIDS conference.
The International Court of Justice (ICJ), the UN's highest legal body, ruled on Friday that the 700 km barrier violated international law.
"Whilst we all accept the government of Israel has a responsibility, and indeed the duty to protect its citizens, any action it takes has to be in conformity with international law and has to respect the interest of the Palestinians," Annan said.
"And Israel, as an occupying power, is responsible for the welfare of the Palestinian people."
Israeli Prime Minister Sharon has
refused the finding of the ICJ
"The report has been given to the general assembly and we'll see where they go from there," he said.
"I don't want to prejudge what they may decide, so we'll leave it to the general assembly."
The ruling said that those sections that cut into Palestinian areas should be torn down and ordered Israel to pay compensation for damages caused.
While the Palestinians hailed the ruling as a great victory, Israeli Prime Minister Ariel Sharon said on Sunday that Israel totally rejected the world court's ruling that the barrier breached international law.
"I want to make clear, the state of Israel absolutely rejects the ruling of the International Court of Justice in The Hague," Sharon said. "It is a one-sided and politically motivated ruling."
"The state of Israel totally rejects the opinion of the world court," Sharon said in his first public reaction to the advisory judgement.
"We decided that it was not wise now to go to the Security Council because we don't want to incite the Americans, especially during the election campaign and it's better to wait until after the elections"
And the White House, Israel's chief supporter, said it was "inappropriate" for the ICJ to issue the ruling, a sentiment echoed by Democrat John Kerry, who is challenging President George Bush in the US presidential election in November.
However, the Palestinian Authority has confirmed it would fight Israel all the way through the United Nations after the court gave its decision by 14 to one.
But on Sunday the PA decided to hold off pushing for a UN resolution against the West Bank barrier until after November's US presidential elections as Israel looked to Washington to veto any eventual vote at the Security Council.
"We decided that it was not wise now to go to the Security Council because we don't want to incite the Americans, especially during the election campaign and it's better to wait until after the elections," one minister said after a meeting chaired by Palestinian leader Yasser Arafat.
"In the meantime we will take the issue to the (UN) General Assembly," he told AFP on condition of anonymity.