The fighters attacked Khak-e-Afghan administrative building in Zabul province at 2030 GMT on Thursday, Zabul deputy governor Mawlawi Muhammad Umar told AFP on Saturday.
One government militiaman was killed in the fighting which lasted three hours. "Seven to eight Taliban bodies were also seen being carried by Taliban fighters early in the morning in nearby villages," Umar said.
The Taliban also kidnapped four close relatives of the district chief and threatened to kill them unless the governor surrendered the district to them, he said.
Zabul province was a former Taliban stronghold and fighters continue to launch attacks from mountain hideouts.
Taliban is back
Suspected Taliban guerrillas last week also attempted to kidnap a US journalists in eastern Afghanistan, but were thwarted when the car they stopped only contained a local driver, a relative of the driver said.
US warns of possible abductions of
journalists by Taliban remnants
After the incident, the US State Department warned that Taliban remnants were planning to abduct American journalists in an attempt to win the release of their compatriots currently held by the United States.
Suspected Taliban fighters have stepped up attacks in recent months in southern and southeastern Afghanistan.