The collapse has shaken financial institutions worldwide and hit big-name investors.
Christopher Cox, the SEC chairman, said in a statement that he had "directed a full and immediate review of the past allegations regarding Mr Madoff and his firm and the reasons they were not found credible".
The SEC "has learnt that credible and specific allegations regarding Mr Madoff's financial wrongdoing, going back to at least 1999, were repeatedly brought to the attention of SEC staff, but were never recommended to the commission for action," Cox said.
"I am gravely concerned by the apparent multiple failures over at least a decade to thoroughly investigate these allegations"
Christopher Cox, SEC chairman
"I am gravely concerned by the apparent multiple failures over at least a decade to thoroughly investigate these allegations."
Cox's statement is sure to fuel a new criticism of the SEC, an agency increasingly seen in Congress and elsewhere as incapable of carrying out its mission - to ensure a basic level of honesty on Wall Street.
An earlier review by the SEC inspector general determined that the agency's monitoring of the five biggest Wall Street firms, which included Bear Stearns, was lacking and Cox himself has come in for strong criticism.
In March, a few days before Bear Stearns nearly collapsed into bankruptcy, Cox told reporters the agency was closely monitoring the five investment firms and had "a good deal of comfort" in their capital levels.
In his statement Cox said that the SEC had not used legal powers to seek information about Madoff's business in the past, but that instead SEC staff "relied upon information voluntarily produced by Mr Madoff and his firm".
The SEC investigation will also "include all staff contact and relationships with the Madoff family and firm, and their impact, if any, on decisions by staff regarding the firm," Cox said.
A growing number of charitable foundations, big international banks and individual investors have acknowledge falling victim to the alleged fraud.
Spain's stock market regulator said investment funds there had direct exposure of $147m, a day after Spain's biggest bank Santander announced potential losses of more than $3bn from Madoff Investment Securities.
Japanese financial firms joined the growing list of those caught up in the scandal, with the Aozora Bank saying its exposure might amount to $137m.
Even the charitable Wunderkinder Foundation, established by Steven Spielberg, the filmmaker, fell victim to the alleged fraud and suffered losses, according to The Wall Street Journal.
Jean-Pierre Jouyet, France's former European affairs minister who this week took over at France's financial markets watchdog, the AMF, said that not for the first time "American regulation is in question".