Dozens of people were arrested and driven away in police vans.
A key focus of protesters' anger was Maricopa County Sheriff Joe Arpaio, a
78-year-old ex-federal drug agent known for his immigration sweeps.
Outside his downtown office, marchers chanted "Sheriff Joe, we are here. We will not live in fear."
Arpaio said he was undeterred and would continue to round up illegal immigrants and turn them over to federal authorities.
|Sheriff Joe Arpaio said he would continue with his sweeps for illegal immigrants [AFP]
"I'm not going to be intimidated and stopped," he said. "If I have to go out and get in the car, I'll do it."
Meanwhile Jan Brewer, the Arizona governor, said she would push ahead with an appeal to a higher court to lift an injunction blocking the most intrusive parts of the law, known as SB 1070.
The appeal comes after US District Court judge Susan Bolton on Wednesday blocked the law's most controversial elements, arguing that immigration matters are the responsibility of the US federal government.
Arizona's governor however has argued that that the federal authorities have failed to secure the border with Mexico and that the state has a right to take matters into its own hands.
In a statement Brewer said she had filed the appeal asking that the suspended provisions "go into effect pending a decision on the merits of this case."
She described the block on key components of the law as a "bump in the road".
The appeal process is likely to take many months and is widely expected to go all the way to the US Supreme Court.
Arizona's Republican-controlled state legislature passed the controversial law three months ago in a move legislators said was an effort to drive nearly half a million illegal immigrants out of the state, and stem the flow of human and drug smugglers over the border from Mexico.
Provisions in the law that have been blocked included one requiring a police officer to check the immigration status of anyone stopped or detained if the officer believed they were not in the country legally.
Immigrants would also have been required to carry their documents at all times and undocumented workers would have been forbidden to solicit work in public.
Measures not subject to the stay, and which went into effect on Thursday, included offenses making it illegal for drivers to pick up day labourers from the street and to
transport or harbour an illegal immigrant.