The neighbours signed a peace accord to end their 1998-2000 war over the border and promised to respect the demarcation of the border set by an independent commission.
But Ethiopia rejected the decision until November 2004, when it said it accepted the commission's ruling in principle but wanted "adjustments".
Eritrea says the Ethiopian position has not changed.
In his latest report to the UN Security Council on the matter, Annan said there had been a "steady increase" of Ethiopian forces near the border area since December and that the deployment appeared to be continuing.
"I am concerned about a possible rise in tensions along the border in view of the build-up," he said.
The UN chief also expressed concern about new construction in areas that were awarded to Eritrea by the boundary commission, which he warned "could be interpreted as an effort to create facts on the ground".
He asked the council to approve another six-month extension of peacekeepers in the UN Mission in Ethiopia and Eritrea until 15 September, and urged an end to the impasse over the border dispute.
"A stalemate in the peace process is a source of instability.
This is exacerbated when troops are amassed in the border region," Annan said, urging both sides to "refrain from any action that could destabilise the situation".
He also recommended another Security Council working trip to both countries as a sign of international support for the peace process.