Rosenthal died of complications from a stroke he suffered two weeks ago, The Times said.
Known as Abe, Rosenthal spent virtually all of his working life at The Times, beginning as a lowly campus stringer in 1943. He rose to police reporter, foreign correspondent, managing editor and finally to the exalted office of executive editor, a post he held for nine years beginning in 1977.
"Abe was a giant among journalists," retired Times publisher Arthur Ochs Sulzberger said in a statement. "He was a great editor with extraordinary loyalty to his troops."
On Rosenthal's watch, The Times published the Pentagon Papers, a history of America's secret involvement in Vietnam, which won the paper one of its many Pulitzer Prizes in 1972.
But the paper started slowly on Watergate and never caught up with the rival Washington Post on the seminal story that brought down a president.
In 1986, facing mandatory retirement, Rosenthal stepped down as editor. Thirteen years later, he was abruptly dismissed, with no explanation, he said, other than a comment by Times publisher Arthur Sulzberger Jr that "it's time".
Rosenthal made clear that the parting was not his idea, telling one questioner that to say he had retired "would imply volition".
When asked by a reporter for the rival Washington Post whether he was fired, he replied, "Sweetheart, you can use any word you want."
The Canada-born Rosenthal was a naturalised US citizen and fervent disciple of American democracy, with an abiding interest in international affairs nurtured by a decade of working abroad.
He covered the UN for eight years from its inception in 1946 and later reported from India, Switzerland, Poland and Japan. His tough coverage of Warsaw's communist regime in the late 1950s earned him expulsion from the country - and journalism awards: the 1960 Pulitzer Prize for international reporting and the first of two Polk awards.
In 1969, taking the helm of a paper in financial distress, Rosenthal, as managing editor and later as executive editor, made sweeping changes to expand advertising and readership.
He beefed up The Times' metropolitan coverage, added a daily business section and specialty sections on sports, weekend features and science and pumped new life into its prose.
In these endeavors, Rosenthal had the backing of the elder Sulzberger, known as Punch, whom he called "probably the best publisher in modern American history".
Though he was acknowledged as a brilliant and incisive editor who, in the words of one Times veteran, "could instantly grasp the essence of any story and how it should be played in the paper", Rosenthal's temperament was less admired. To some detractors he was an overbearing tyrant, whose autocratic behaviour could derail a promising career.
RW 'Johnny' Apple, a top Times reporter for four decades, was quoted in a 1999 magazine article as saying Rosenthal "was not the nicest man to work for, but - and there's a major but - he may have saved The New York Times".
The Times credited Rosenthal with being "instrumental in mustering the arguments" in favour of publishing the explosive Pentagon Papers, which revealed how four successive US administrations had become enmeshed in Indochina.
Rosenthal's famous newsroom maxim, "keep the story straight", was prompted in part by the conservative-minded editor's perception of a leftist tilt in the paper's coverage, but some Times colleagues thought he overcompensated by giving too little attention to such stories as strife-torn Central America and Aids.
As editor, Rosenthal barred the use of "Ms" or the word "gay" in reference to homosexuals - a far cry from The Times' later emphasis on diversity. He also began the paper's practice, now imitated by many others, of running corrections as a prominent daily fixture.
Rosenthal stepped down as executive editor in 1986 at 64, a year short of the paper's mandatory retirement age, and began a twice-weekly op-ed column.
Rosenthal was four when his parents, Harry and Sarah, moved the family from Canada to the US. He grew up in the Bronx and began his newspaper career in 1943 as a campus stringer for The Times while attending City College of New York.