The shake-up, the most extensive since Karzai assumed leadership in 2001, affects Ustad Atta Mohammad and General Abd al-Rashid Dostum, besides governors and police chiefs of four provinces loyal to them.
Atta is the commander of a military corps while Dostum has been serving as Karzai's adviser in security and military affairs.
The revamp follows a security meeting earlier this week in Kabul and intense fighting between Atta and Dostum's supporters in the north, earlier this month.
Karzai faces the dual challenge of controlling military commanders allied with him and countering the threat posed by regrouping Taliban fighters.
Atta said he and Dostum would be given new positions in the central government after the shake-up.
"We will be moved to Kabul. We will be based in Kabul, but it is not yet clear what positions we will be given," he said.
The governors and police chiefs of Sari Pul, Balkh, Samangan and Jszjan provinces may also be given new positions.
Though Atta welcomed the impending revamp, Dostum's reaction is not yet known.
Despite being on the side of Karzai, Atta and Dostum run their own private armies and fight intermittently among themselves.