According to a local police chief a bomb exploded in a tourist bus on Thursday just minutes after Manmohan Singh ended a meeting in Indian Kashmir's main city of Srinagar aimed at restoring peace.
The explosion happened despite massive security in Srinagar for the talks.
Three tourists from the western Indian state of Gujarat were killed on the spot and doctors said a fourth died in hospital.
A doctor said an infant was among the dead and that two other people were in a critical condition.
No one has claimed responsibility for the attack.
Meanwhile, Singh said he was upbeat after the talks despite a boycott by the Kashmiri separatist group All Parties Hurriyat Conference.
The group, which seek the region's merger into Pakistan or the creation of a separate nation, said it will not negotiate with pro-India groups but is willing to meet Singh separately.
Singh, who met them in New Delhi in February, declined to do so this time.
Security has been high in
Srinagar during the talks
He said his country was committed to living in harmony with neighbouring rival Pakistan and to resolving the issue of Kashmir.
He said if violence in the region reduced, then the number of troops, currently estimated at more than half a million, could do likewise.
Singh proposed several measures to encourage interaction among Kashmiris divided by the India-Pakistan border, including creating simpler travel procedures.
The prime minister also admitted human rights abuses are committed by Indian armed forces.
"Our armed forces are not armed forces of occupation, they are there to protect ... our citizens. They have a very sound record but it does happen," he said.
"I cannot deny sometimes aberrations do take place but these aberrations cannot be allowed to become a norm."
Separatist groups had threatened to sabotage the two-day talks and called a general strike that shut shops, businesses and schools for a second consecutive day on Thursday.
But at the talks Omar Abdullah, president of the main opposition National Conference, was more conciliatory, proposing that Kashmir get more autonomy within the Indian union something midway between the separatists' and pro-India groups' demands.
"I cannot deny sometimes aberrations do take place but these aberrations cannot be allowed to become a norm"
Indian prime minister
The Kashmir talks also come alongside a wider peace process between India and Pakistan.
"I have a vision that the peace-making process must ultimately culminate in our two countries entering into a treaty of peace, security and friendship," Singh said.