The protesters shouted for the resignation of Jose Luis Rodriguez Zapatero, the socialist prime minister.
"I used to think Zapatero was an exciting character, with purpose and ideas, but now, no way"
Asuncion Casanova, business consultant
Asuncion Casanova, a business consultant, said: "This is the last straw. We're fed up with this man trying to dismember Spain."
"I used to think Zapatero was an exciting character, with purpose and ideas, but now, no way."
Opponents of the government have taken to the streets on many occasions in the past year, but Saturday's protest is the first to be organised by the Popular Party.
On Friday night thousands attended smaller PP protests in regional capitals across Spain.
PP leaders have recently verbally attacked Zapatero while laying flowers at the spot where Inaki de Juana Chaos, the prisoner in question, killed 12 police in a 1986 car bombing.
De Juana was moved to a hospital in the Basque Country last week after being chained to a Madrid hospital bed and force-fed during a 114-day hunger strike.
Once he recovers sufficiently, he will serve the remainder of an 18-month sentence at home.
He has already served 18 years for 25 murders but went on hunger strike after being given a new term for making threats.
The government says de Juana would have starved himself to death if he had not been moved, and on Friday Zapatero said his government would not change its decision.
Opinion polls in both left and right-leaning newspapers show a slim majority of Spaniards think the government was wrong to grant house arrest to de Juana.