The state government of Uttar Pradesh, where Agra is located, issued a warning, asking tourists to stay in their hotels until order was restored.
Udai Ram, another local government official, said that the Taj Mahal was outside of the curfew zone and was open to tourists, but that one of its two gates had been closed closed.
Vikram Singh, the director-general of police in Lucknow, the capital of Uttar Pradesh state, said the rioting had been sparked by residents who had gone "berserk after a speeding truck killed four people".
A senior official in the Agra district said: "Four died in a road accident earlier in the day and one more was killed in violence that broke out later."
The four were walking home from a festival that falls about two weeks before the start of Ramadan, the month of fasting for Muslims.
News of the accident, which occurred about four kilometres from the Taj Mahal, spread and, according to witnesses, a crowd had begun to confront the police.
Hotels and tour operators organising visits to the Taj Mahal, built by the emperor Shah Jahan in the 17th century as a tomb for his queen, have said that Indian and foreign visitors were safe.
Mukesh Rai, who organises visits to the mausoleum said: "Some visitors from our hotel had been to the Taj this morning but they have all come back safe."
"There is one way still open and three of our visitors who had gone came back very soon - well before the curfew was declared."
Sunil Sharma, who works at a nearby hotel said that half a dozen visitors from the hotel who toured the monument earlier on Wednesday had also returned safely.
Sharma said: "I think things are settling down."
However, an official from the Mughal Sheraton, a hotel 1.5km from the Taj Mahal, said that it had received many cancellation requests since Wednesday morning.
The official said: "The main motive of people visiting Agra is to see the Taj, and when they heard about the curfew, I think people preferred to cancel."
Almost three million domestic and foreign tourists visit the Taj Mahal annually, more than any other tourist site in India.
In July, it was voted as one the world's top new "Seven Wonders" through a mobile phone text message and Internet poll.