"We've heard from high-level sources that this would be a radical reduction in operations.
"The UN staff would be reduced in all agencies, not just the political one - Unama, but the world food programme, the refugee agency, UNHCR.
"It will send a big signal to all the other NGOs that are working in this country, to review their security situation."
News of a withdrawal followed an unannounced visit by Ban to the Afghan capital where he urged Hamid Karzai, the Afghan president, to beef up security for UN staff working in the country.
The suicide attack, which left five UN employees dead, targeted the Bekhtar guest house in central Kabul in an area close to several government ministries.
The three Taliban attackers and four Afghans were also killed during a fierce gun battle that broke out after the raid.
In a statement afterwards, the Taliban said it had attacked the UN because of its involvement in organising Afghanistan's presidential elections, the second round of which has since been called off.
On Wednesday, the UN announced it was taking additional steps to reduce risks to its national and international staff in Afghanistan in light of the attack and "ongoing threats".
A UN statement said the new measures would involve short-term relocations for some staff while additional security was put in place.
The UN Assistance Mission in Afghanistan (Unama) is composed of roughly 1,100 foreign personnel working across the country, in addition to about 5,600 local staff.
Their duties range from providing security and co-ordinating aid to monitoring human rights conditions.
Our correspondent said the UN is planning to set up a new hub in Dubai. But he said the scaling back of the world body's mission in Afghanistan could be an "explosive decision" because the international staff is necessary to make things works on the ground.