In August, at least 35 people were killed in Orissa after the death of a hardline Hindu priest and four of his followers sparked violence between Hindus and Christians.
Indian Maoists claimed responsibility for killing Swami Laxmananda Saraswati, saying he was forcing tribal people to reconvert to Hinduism.
They also claimed that the state government had "made it look like Christian groups [were] responsible for the attack".
But Hindu hardline groups rejected the Maoist claim, saying Saraswati opposed conversions to Christianity and his elimination could only benefit Christian missionaries active in the area.
In India's northeastern Assam state, 50 people were killed in clashes between Muslim migrants and tribal groups earlier this month.
|Violence between different religious groups have flared in several states [EPA]
The prime minister's warning came as police imposed a curfew in the southern Indian state of Andhra Pradesh two days after the latest clash between Muslims and Hindus which left three people dead.
The country has also been rocked by a series of bomb blasts targeting major cities this year which killed more than 100 people killed.
A home-grown Islamic group, the Indian Mujahideen, claimed responsibility for the attacks in Jaipur, Bangalore, Ahmedabad and New Delhi, saying they were in revenge for attacks on Muslims across India.
Singh said in his speech that "there can be no compromise with terrorism, and terrorists have to be dealt with firmly".
"We need to meet today's mindless violence with the requisite amount of force but must also ensure that this is tempered by reason and justice which is the normal order of governance," he added.
India, which is majority Hindu with a large Muslim minority, is officially secular.