Speculation that Souter, 69, was planning to retire started after he failed to appoint clerks for the court term that starts in October.
Souter is seen as having Republican roots and as disliking Washington. He once described his situation as "the world's best job in the world's worst city".
An unmarried liberal intellectual, he upheld abortion rights in 1992 and voted to support the government's right to seize property to foster economic development.
Souter nor the White House have yet commented on his possible departure.