He admitted to one of the crimes with a hint of pride in his voice and a wink at prosecutors on Wednesday.
Rudolph, 38, entered his pleas during back-to-back court appearances - first in Birmingham, Alabama, in the morning, and then in Atlanta in the afternoon - after working out a plea bargain that will spare him from the death penalty.
He will get four consecutive life sentences without parole.
The four blasts killed two people, including a police officer, and wounded more than 120 others.
When asked in Atlanta whether he was guilty of all the bombings, Rudolph politely and calmly responded: "I am."
"The purpose of the attack on July 27th  was to confound, anger and embarrass the Washington government in the eyes of the world for its abominable sanctioning of abortion on demand," Rudolph said in a statement, which quoted the Bible throughout.
"I am not anarchist. I have nothing against government or law enforcement in general. It is solely for the reason that this government has legalised the murder of children that I have no allegiance to nor do I recognise the legitimacy of this particular government in Washington."
The downtown Atlanta courthouse where Rudolph entered his pleas is two blocks from Centennial Olympic Park, where a bomb hidden in a knapsack exploded and sent nails and screws ripping through a crowd at the height of the 1996 Atlanta Games.
A woman was killed and 111 other people were wounded.
"The purpose of the attack on July 27th  was to confound, anger and embarrass the Washington government in the eyes of the world for its abominable sanctioning of abortion on demand"
Rudolph said he had planned a larger attack on the Olympics that would have used five bombs over several days.
He said he planned to make phone calls before each explosion, "leaving only uniformed arms-carrying government personnel exposed to potential injury." But he said poor planning on his part made that five-bomb plan impossible.
"I had sincerely hoped to achieve these objections without harming innocent civilians," he said. He added: "There is no excuse for this, and I accept full responsibility for the consequences of using this dangerous tactic."
He said he blew up four other bombs in a vacant lot in Atlanta and left town "with much remorse".
In the Atlanta courtroom, as prosecutors read details about the bomb that killed 44-year-old Alice Hawthorne at the Olympics, Hawthorne's daughter, Fallon Stubbs, 22, crossed her arms over her chest and looked at her feet.
Hawthorne's widower, John, rocked slightly and covered his head with his hands. Other family members wept.
Rudolph avoided the death
penalty but was jailed for life
Afterwards, Stubbs described the day as "exhausting, to say the least" and said she would address the court at Rudolph's sentencing.
"It'll be my time to get it out," she said.
Richard Jewell, the security guard who was initially hailed as a hero for helping evacuate the park just before the blast, but was later reported to be under FBI investigation, was also in the courtroom but refused to comment on the plea.
Jewell was eventually cleared by the FBI and now works as a police officer.
Rudolph also admitted bombing a gay nightclub in Atlanta, wounding five people, in 1997, and attacking a suburban Atlanta office building containing an abortion clinic that same year.
Six people were wounded in that attack, which consisted of two blasts, first a small one to draw police, then a larger explosion.
Rudolph used a nail bomb in his
Atlanta Olympics attack
In Birmingham earlier, Rudolph pleaded guilty to an abortion clinic bombing there that killed an off-duty police officer and maimed a nurse.
A defiant Rudolph winked towards prosecutors as he entered court, and during the hearing he said the government could "just barely" prove its case if it had gone to trial.
Thought to be a follower of a white supremacist religion that is anti-abortion, anti-gay and anti-Jewish, Rudolph hid for more than five years in the mountains of western North Carolina, apparently using the survival skills he learned as a soldier.
He was captured in Murphy, North Carolina, in 2003, scavenging for food behind a grocery store, after becoming something of a folk hero to some people in the countryside for his ability to elude a manhunt by the government.
As part of the plea agreement, Rudolph told the authorities where to find more than 112.5kg of dynamite buried in North Carolina. The government said some of the explosives were near populated areas and could have become unstable and blown up.
Deborah Rudolph, Rudolph's former sister-in-law, said being kept in solitary confinement with only one hour a day of fresh air is a fitting punishment for an outdoorsman who hated the government.
"Knowing that he's living under government control for the rest of his life, I think that's worse to him than death," she said from her home in Nashville, Tennessee.