The Federal Bureau of Investigation ruled out terrorist links with the crash on Wednesday.
The plane, a Cirrus SR20, was being flown by New York Yankees pitcher Cory Lidle, 34, who was one of the two dead.
The Federal Aviation Administration (FAA) said the small aircraft hit the building at East 72nd street at about 2.45 pm (1845 GMT).
The military scrambled fighter jets above US cities as a precaution, said Admiral Tim Keating, commander of the US Northern Command.
George Bush, the US president, was informed shortly after the crash.
"I just stood there in shock, I thought 'this can't be happening to us again"
New York resident
A woman who lives on the same street, Chris Foege, said: "I just stood there in shock, I thought 'this can't be happening to us again."
It was like "9-11 all over again."
"It's a small general aviation aircraft," Les Dorr, spokesperson for the Federal Aviation Administration, told AFP.
The aircraft is believed to have hit the 20th floor of the building.
Live television coverage on US cable networks showed black smoke and flames pouring from several rooms on two upper floors of the building.
The television images of the burning building evoked memories of the devastating attacks of September 11, 2001, on New York and Washington when hijacked airliners were flown into the World Trade Centre and the Pentagon, killing almost 3,000.