Shakeel Ahmad, Delhi's home minister, told Al Jazeera "the number of casualites may go a little higher, because the number of people admitted to different hospitals is still not known".
The so-called Indian Mujahidin group, which also said it was responsible for a string of five bomb attacks in July in the western commercial city of Ahmedabad that killed at least 45 people, made the claim in an email to India's NDTV, the Press Trust of India said.
NDTV quoted the email as saying: "In the name of Allah, the Indian Mujahidin has struck back again."
Little is known about the group who also threw down a challenge to India's security agencies in the email, saying "do whatever you want and stop us if you can".
Matt McClure, Al Jazeera's correspondent in Delhi, said: "The first blast happened at 6.15pm in Karol Bagh, a busy market in central Delhi."
One bomb in Karol Bagh had been planted in a rickshaw and another in a scooter behind it, Indian media reported.
McClure said: "A separate blast hit Connaught Place, the central business district, including a blast within a rubbish bin in central park, which at this time would have been full of thousands of people."
Twenty people were reported to have been injured in the explosion at the Gopaldas Bhawan car park.
"We heard a very loud noise, then we saw people running all over the place," Chanchal Kumar, a witness whose shirt was soaked in the blood of several victims he had helped carry into ambulances, said.
"There were about 100-200 people around this place," he said.
A blast on Barakhamba Road then went off near the metro station, after which twin blasts were reported in the Greater Kailash market.
Bomb disposal squads rushed to the bomb-sites which were cordoned off.
Authorities turned off mobile phone networks amid fears phones could be used to trigger more blasts.
The bomb attacks come after similar attacks in Jaipur, Bangalore and Ahmedabad.
The neighbouring states of Punjab and Haryana as well as all major Indian cities have been put on maximum alert.