Bomb-disposal experts alerted by a railway employee on Friday found 10 to 12kg of explosives, possibly dynamite, under a track about 60km south of Madrid on the line to the southern city of Seville, Interior Minister Angel Acebes said.
The explosives were connected to a detonator with a 136m cable, the minister told a news conference on Friday.
He said it was not immediately known who placed the bomb.
"As we get information regarding those possibly responsible or details that move the investigation forward, we will give them to you," Acebes said.
Friday was a busy travel day in Spain, with trains and highways full of people leaving home for the Easter vacation.
The state rail company, RENFE, said no train was close to the bomb when it was detected.
The line serves mainly Spain's AVE high-speed trains, which have a maximum speed of 300km per hour, but a smaller number of slower trains also use it, RENFE said.
The discovery of the bomb came less than a month after 10 backpack bombs ripped through four commuter trains in Madrid, killing 191 people and injuring more than 1800.
The focus of that investigation is a Moroccan group with alleged links to al-Qaida. The bombs were detonated with cell phones attached to the explosives.
The Basque separatist group ETA has also targeted Spanish rail lines in the past.
Also on Friday a judge charged a Moroccan man with involvement in the 11 March bombings making him the 15th person accused in connection with the atrocity.
The judge accused the man of "belonging to or collaborating with a terrorist organisation".
The 11 March bombings claimed
191 lives and wounded 1500
The man was a labourer at the house where the bombs were prepared and told the court he was asked not to show up for work during the week of the attacks, a court official said.
But High Court Judge Juan del Olmo also released four other suspects.
Nine detainees have been released or cleared of accusations in the bombings, although some of them remain under watch and one is in jail for a separate case.
Five detainees are accused of the murders and attempted murders, indicating they are believed to have played a central role in the bombings.
The others are suspected of collaborating in some lesser role. Formal charges will come later.
On Thursday, police in northern Spain defused three letter bombs addressed to journalists in Madrid.
Acebes said the origin of the letters has not been determined, although the mechanism of the explosives is "similar to those that have been used by anarchist groups on previous occasions".