Local residents called in to state television saying they had felt the explosion. However, a US embassy spokesperson said that no injuries had resulted from the blast.
The senior police official said Greece's deputy police chief and Athens police chief had gone into the building together with officers of the national security and anti-terrorist squads.
Dozens of police cars surrounded the embassy and police cordoned off all roads in the area.
US embassy is surrounded by a three metre-high steel fence, with guards posted at every entrance and at street corners around it.
Attacks
Greece's biggest domestic security threat, the left-wing November 17 group, which killed US and other foreign diplomats in Greece, was dismantled in 2002.
In November last year, Greek riot police fired tear gas to disperse demonstrators marching to the US embassy in Athens who chanted slogans including "Bush the butcher, out of Iraq" and "The USA is the real terrorist".
Reporting from Athens, Barnaby Phillips, Al Jazeera's correspondent, said: "In February 1996, unidentified assailants fired a rocket at the US embassy compound in Athens, causing minor damage to three diplomatic vehicles and some surrounding buildings."
He said November 17 was responsible for 20 murders, including those of several American officials.
"This was an act of powerful symbolism," Phillips said. "Now the Americans must put their faith in the Greeks to find those responsible. Otherwise there will be fears of a return to the dark days of political extremism."