Minutes before al-Sistani's arrival on Thursday, interim Prime Minister Iyad Allawi said he had ordered his forces to observe a 24-hour ceasefire in Najaf to allow the negotiations to take place.
He also offered an amnesty deal to besieged al-Mahdi Army fighters and safe passage for their leader al-Sadr.
US commanders said they too were suspending offensive operations in Najaf.
American and Iraqi troops have been besieging the militiamen holed up in Najaf's renowned Imam Ali mosque for three weeks. Allawi had previously warned al-Mahdi Army fighters to surrender, or be defeated in battle.
Talks between representatives of the Ayat Allah and aides of al-Sadr have begun, said the former's spokesman.
Backed by thousands of supporters and accompanied by British occupation troops and Iraqi security forces, al-Sistani set off on the 400km journey north from Basra on Thursday morning.
But within hours of al-Sistani's arrival, at least 10 Iraqis, who had answered his call to travel to Najaf and "rescue" the city from the fighting raging there, were shot dead by unidentified assailants.
The killings in Najaf followed two deadly attacks earlier in the day in the neighbouring city of Kufa that left more than 70 people dead.
More than 70 people have been
slaughtered in Kufa
A mortar attack on a Kufa mosque used by al-Sadr and his supporters was followed by another deadly assault when al-Sistani supporters bound for Najaf came under fire.
The firing came from a base between the two cities housing national guardsmen and US troops, witnesses said.
In Baghdad, Allawi said al-Sadr's representatives had signalled they would accept a peace plan announced by al-Sistani.
Prime Minister Iyad Allawi has
offered 'safe passage' to al-Sadr
He said al-Mahdi Army fighters would receive an amnesty if they gave up their weapons and agreed to leave their besieged position peacefully.
Heavy bombardment of the area around the highly revered mosque by US occupation forces, and al-Mahdi Army resistance fighter's rejection of government ultimatums to surrender and withdraw, has presented Allawi with a challenging crisis.
Al-Sadr – whose whereabouts remain a mystery - would also be offered safe passage if he agreed to end the uprising, Allawi said.
The ceasefire would take effect from 3pm (1100 GMT), Allawi said.
Earlier, fierce fighting raged around the mosque in Najaf overnight on Wednesday and into Thursday morning as US-led air and ground forces tightened their grip around the religious site.
American snipers had taken up positions in buildings around the mosque, preventing those inside the mosque from leaving easily.
The Iranian-born al-Sistani, Iraq's most senior Shia Muslim figure, arrived in via the Kuwaiti border on Wednesday after a three-week spell in London.
He had gone to Britain for medical treatment for a heart problem last month, just as the Najaf fighting erupted.
The Ayat Allah has previously told Iraqis not to take up arms against occupation forces and advised them to wait for elections to take place.