Annan chose the Truman museum for his final big speech in part because it is dedicated to a president who was instrumental in the founding of the United Nations. His text repeatedly praised the Truman administration but never mentioned the current office-holder, George Bush, by name.
Annan said: "As President Truman said, the responsibility of the great states is to serve and not dominate the peoples of the world.
"He believed strongly that henceforth security must be collective and indivisible. That was why, for instance, that he insisted when faced with aggression by North Korea against the South in 1950, on bringing the issue to the United Nations.
"Against such threats as these, no nation can make itself secure by seeking supremacy over all others."
"As President Truman said, the responsibility of the great states is to serve and not dominate the peoples of the world."
Kofi Annan,Un secretary-general
Annan also called for a reform of the Security Council, saying its membership "still reflects the reality of 1945".
He suggested adding new members to represent parts of the world with less of a voice.
He said the permanent members, the world powers, "must accept the special responsibility that comes with their privilege".
"The Security Council is not just another stage on which to act out national interests," he said.
During his two five-year terms as UN leader, Annan has tangled often with Bush's administration, particularly over the invasion of Iraq in March 2003, launched without consent from the UN Security Council.
He said: "More than ever, today, Americans, like the rest of humanity, need a functioning global system through which the world's peoples can face global challenges together. And in order to function, the system still cries out for far-sighted American leadership, in the Truman tradition."
Bush administration officials have argued that Washington should use the UN only to serve its national interest.
Annan said: "None of our global institutions can accomplish much when the US remains aloof. But when it is fully engaged, the sky's the limit."
Annan steps down at the end of the month, to be succeeded by Ban Ki-Moon of South Korea.
A multilateral century
As Washington reviews its policies in Iraq, Annan has pushed for greater involvement by Syria and Iran, a more inclusive political system and greater human rights protections.
"Kofi Annan had been a bit too soft during his tenure and the global issues are still un-resolved and lingering,"
Sohail Gill, Pakistan
Send us your views
Truman, who ordered two atomic bombs dropped on Japan in 1945, making the US the sole power in history to use nuclear weapons, learnt from that experience that security from then on "must be collective and indivisible", Annan said.
He said: "You Americans did so much, in the last century, to build an effective multilateral system, with the United Nations at its heart. Do you need it less today, and does it need you less, than 60 years ago?
"When power, especially military force, is used, the world will consider it legitimate only when convinced that it is being used for the right purpose, for broadly shared aims, in accordance with broadly accepted norms."
Annan said that the US has historically been a leader in human rights.
"When it appears to abandon its own ideals and objectives, its friends abroad are naturally troubled and confused," he said in an apparent reference to charges of abuse at US prisons in Guantanamo Bay, Cuba, and Iraq's Abu Ghraib.
Annan hit out at Washington's opposition to expansion of the 15-nation Security Council as part of a reform drive.
He said: "It is only through multilateral institutions that states can hold each other to account.
"And that makes it very important to organise those institutions in a fair and democratic way, giving the poor and the weak some influence over the actions of the rich and the strong."