The Mali Blues great died in his homeland, Mali, where radio stations suspended regular play to send the Mali bluesman's signature lilting sounds out over the airwaves.
Toure, in his late 60s and known to be battling cancer, melded traditional Malian stringed instruments and vocals with the American blues guitar work he considered firmly rooted in West Africa, from where most North American slaves were shipped.
Best known overseas for his 1994 collaboration with American guitarist Ry Cooder on Talking Timbuktu, which netted him his first Grammy, Toure was a local giant who helped bankroll a production company that has nurtured Mali's young talent.
Malick Konate, commercial director of the label Mali K7, said: "His death is a great loss for Mali and also for Africa. We can only pay him homage."
Toure, one of Africa's best-known artists, won his second Grammy this year in the traditional world music album category for In the Heart of the Moon, performed with fellow Malian Toumani Diabate.
The Mali Culture Ministry said Toure died early on Tuesday at his home in the capital, Bamako, after a long struggle with an unidentified illness.
Mbaye Boubacar Diarra, a television producer, said: "A monument has fallen. With the death of Ali Farka Toure, Mali is losing one of its greatest ambassadors."
Djeneba Seck, a singer, said: "I'm completely in mourning. It's as if I lost my father."
Toure was born in 1939 in the northern Sahara Desert trading post of Timbuktu. Like many Africans of his generation, the exact date of his birth was not recorded.
Toure learned the traditional single-stringed gurkel at an early age, later also taking up the guitar, at which he was also accomplished. He cited many Western musicians for inspiration, including Ray Charles, Otis Redding and John Lee Hooker.
Toure believed American blues
was rooted in West Africa
Toure spent much of his later years in his childhood town of Niafunke, near Timbuktu, which has become a pilgrimage spot for many music-loving Africans and tourists seeking one of the original progenitors of a genre known as Mali Blues.
In 2004, Toure became the small farming town's mayor.
His family said although no date had yet been determined, he will be laid to rest in the sandy loam near Timbuktu.
"For some people, Timbuktu is a place at the end of nowhere," he once was quoted as saying. "But that's not true, I'm from Timbuktu, and I can tell you that it's right in the centre of the world."