Police said they had rounded up almost 90 people in the wake of Wednesday's attacks, which occurred within an hour of each other in 64 towns and cities.
"One of them has confessed that he was Jamayetul Mujahideen's local leader and that he carried out an attack here," said Abdur Rahim, superintendent of police in Satkhira district, where two people were detained.
"We are still questioning both men to get an idea about this organisation," Rahim said on Thursday.
The Home Ministry said leaflets bearing the name of Jamayetul Mujahideen were found at all the blast scenes.
Jamayetul Mujahideen and another hardline group, Jagrata Muslim Janata Bangladesh, were banned in February for alleged links to a series of bombings of non-governmental groups, religious shrines and other targets.
Prime Minister Khaleda Zia was in
China at the time of the blasts
"Whoever did this made a big statement. It shows how fragile the security situation is here," said security expert Sakhawat Hossain, a former army Brigadier General.
"In terms of security, it shows that we are now the weakest country in South Asia, even weaker than Nepal," he added.
Security was immediately stepped up across Bangladesh in the wake of the attacks, and a number of arrests were made.
Security problems hurt Bangladesh earlier in the year when a February summit of South Asian nations in Dhaka was postponed after India pulled out after a grenade attack on an opposition party rally that killed five people.
In May, neighbouring India said it was concerned at what it called the rise of Islamic extremism in Bangladesh.
"In terms of security, it shows that we are now the weakest country in South Asia, even weaker than Nepal"
Ex-army Brigadier General
The bombings began 90 minutes after Prime Minister Khaleda Zia left Dhaka for a five day visit to China. It was not known whether she would cut short her trip.
Zia, whose Bangladesh Nationalist Party (BNP) leads the country's coalition government, condemned the attacks as "cowardly" acts of terrorism, the official BSS news agency said.
The opposition Awami League said it was calling a day-long nationwide strike on Saturday to protest against the bomb attacks.