Fayyad had issued a fatwa (decree) a few days ago urging people not to follow orders of Taliban leader Mulla Muhammad Omar, Aljazeera's correspondent said.
Maulvi Fayyad was killed in front of his office by armed men on Sunday, deputy police chief General Muhammad Salim said.
"Two armed men riding on a motorcycle opened fire on Maulvi Fayyad when he was standing in front of his office," Salim said.
He was known as a close supporter of US-backed President Hamid Karzai.
The Taliban has claimed responsibility for the attack, according to an AFP report.
The AFP report quoting Taliban spokesman Mullah Abd al-Latif Hakimi said: "Today at 2 in the afternoon Taliban shot and killed Mullah Fayyad in Kandahar city."
Police said they had launched an investigation.
"The investigation is going on and we don't know now who was behind the attack," said Salim.
Aid worker video
Meanwhile, a videotape of an Italian aid worker kidnapped in Afghanistan nearly two weeks ago was broadcast by an Afghan television station on Sunday.
Clementina Cantoni was
abducted on 16 May in Kabul
Wearing a headscarf and wrapped in a brown shawl, Clementina Cantoni, 32, was shown flanked by two men pointing assault rifles at her in the tape broadcast by private Tolo TV.
She was asked to name some close family members by a man who was not seen on the tape. She did not make an appeal for her freedom.
The men pointing rifles at her had scarves across their faces and ammunition pouches on their chests.
Tolo did not say how it got the tape.
Cantoni, who works for Care International, was abducted on 16 May by four armed men who stopped her vehicle on a street in the centre of the capital.
Afghan government officials were not available for immediate comment, but in Rome the Foreign Ministry's press office said the video "appeared authentic and calming"."Contacts continued" with the kidnappers, the office added, without elaborating.