Hundreds of armed men marched through the capital of Anbar province as mosque loudspeakers broadcast a statement by the Mujahidin Shura Council.
Cars also formed convoys in a show of strength in an area revisited by US forces on a regular basis.
"We are from Mujahidin Shura Council and our amir is Abu Omar al-Baghdadi. God willing we will set the law of Sharia here and we will fight the Americans," said a man who identified himself as Abu Harith, a Mujahidin leader.
"We have announced the Islamic state. Ramadi is part of it. Our state will comprise all the Sunni provinces of Iraq," he told the Reuters news agency.
The group often claims responsibility for attacks against US-led forces and the government in Baghdad.
Last week, the Shura Council announced the formation of the Mutayibin Coalition (a reference to a pre-Islam pact in which the leaders of Prophet Muhammad's tribe agreed that they would not let each other down) pact to step up the fight against US-led forces and urged Sunni Muslim tribal leaders to join.
The coalition called for a separate Islamic state "to protect our religion and our people, to prevent strife and so that the blood and sacrifices of your martyrs are not lost".
"We have announced the Islamic state. Ramadi is part of it. Our state will comprise all the Sunni provinces of Iraq"
Mujahidin Shura Council leader
Abu Harith said the state would be headed by Amir Abu Omar al-Baghdadi, a little-known militant, and would include the Sunni areas of Baghdad, the provinces of Anbar, Diyala, Kirkuk, Salah al-Din, Nineveh and parts of Babil and Wasit. Iraq has 18 provinces.
In Anbar, which makes up a third of Iraq's territory and is an area loyal to fighters opposed to the US-led forces, some tribal leaders have vowed to take on al-Qaeda, which they accuse of imposing a harsh interpretation of Islam.
One man, who refused to give his name, told Reuters that Ramadi was just the beginning and the rest of the provinces would follow. He said the Sunni Islamic state was a response to a decision last week by Iraq's parliament to pass a law that allows provinces to form federal regions. He said the law weakened the Sunnis.
Sunni Arab political parties oppose federalism saying it will lead to the break-up of Iraq.
"We need to secure the rights of the Sunnis based on God's sharia," he said. "Who will dare to oppose God's sharia?"
Abu Harith said Sunni Arabs were being marginalised.
"We are all Iraqis and we will fight for the rights of the Sunnis. We want to bring justice back to its people."