"I can truthfully say that I regret having made such remarks," Williamson said in his statement, posted on a Catholic news website.
He said he would never have made them if he had known "the full harm and hurt to which they would give rise".
"To all souls that took honest scandal from what I said, before God I apologise," he said.
But Williamson did not say his comments were incorrect, or that he no longer believed them.
The Vatican had demanded that the bishop "absolutely and unequivocally distance himself" from his remarks about the Holocaust.
Lombardi said that Williamson's apology was not addressed to Pope Benedict XVI or to the Vatican's Ecclesia Dei commision, which had been dealing with the Society of St Pius X, the order the bishop belongs to.
The pope had lifted Williamson's excommunication, along with that of three other bishops, in January to in order to help bridge a divide with traditionalists that occurred after liberal reforms in the 1960s.
The Vatican later said that Williamson's comments were not known to the pope when he lifted the excommunication.
Williamson's apology also triggered anger from Jewish groups.
Charlotte Knobloch, the head of Germany's central council of Jews said: "With his failure to clearly retract his malicious lies, Williamson has shown again that he is a convinced anti-Semite and an incorrigible Holocaust denier".
Elan Steinberg, vice president of the American Gathering of Holocaust Survivors and their Descendants, said he could not tell if Williamson's apology was genuine.
"If it is, let him reflect over the coming weeks and make a proper act of penance,'' he said.
"For our part, we seek to move ahead and resume the Catholic-Jewish dialogue with renewed vigor and determination."