They want Tripoli to settle what they called "remaining terrorism cases", including unresolved compensation for US relatives of victims of the 1988 bombing of PanAm flight 103 over Lockerbie, Scotland.
The 24-hour visit on Wednesday was aimed principally at discussing the crisis in Sudan's Darfur region.
Negroponte said Washington wanted to build a new embassy and appoint an ambassador to continue the improvement in bilateral ties since Libya ended a mass destruction weapons plan in 2003, a move that helped end its long international isolation.
Officials declined to speculate why Negroponte left without meeting Gaddafi, who has met a string of other US official visitors over the past four years including state department diplomats.
US senators, including Democratic presidential candidate Senator Hillary Clinton, had urged Negroponte to hold Gaddafi accountable for "acts of terrorism" on his visit.
Gaddafi told supporters at a ceremony last week to mark the 21st anniversary of the 1986 US attacks on Tripoli and Benghazi that Libya had resolved the Lockerbie case and completed compensation.
Negroponte, who visited Tripoli as part of a wider African tour, also raised the case of six jailed foreign medical workers.
Washington backs EU member Bulgaria's demand that Tripoli free five Bulgarian nurses and a Palestinian doctor sentenced to death for deliberately infecting hundreds of children with HIV at a hospital in Benghazi in the 1990s.
Sofia insists that the six medics are innocent. Libyan authorities dismiss this, saying they are part of Western pressures to win more concessions from Tripoli.
The Bush administration restored formal ties with Tripoli in May 2006, opening an embassy for the first time in more than 25 years.