Following the delivery of the inventory or "declaration" on Thursday, North Korea demolished the Yongbyon cooling tower, allowing foreign media to broadcast the event.
Pyonyang is disabling Yongbyon under US supervision in return for energy aid and the lifting of some sanctions.
"In the next phase we do have to move on abandonment. That is the purpose of six-party talks," Rice said.
Sung Kim, the US envoy to North Korea, also speaking in Seoul on Saturday, said North Korean engineers appeared to have formed an "emotional attachment" to their atomic programme.
He had seen emotion in the eyes of Ri Yong-ho, the head of the Yongbyon nuclear reactor, and his colleagues when they all witnessed Friday's demolition of the plant's cooling tower.
"I think I detected ... a sense of sadness when the tower came down but I thought he [Ri] put it well when he was asked what this all meant for him and he said that he just hoped this would contribute to peace and stability," Sung said.
Rice said she and Yu had held extensive talks about efforts to verify the North Korean nuclear inventory, which covers nuclear facilities and the production of bomb-making plutonium but not weapons.
Critics point out that it also fails to address concerns about a suspected secret highly enriched uranium weapons (HEU) programme or deal with suspicions that North Korea was involved in nuclear proliferation to Syria though both are referred to in separate documents.
"There are documents that are referred to in the declaration concerning those two issues - HEU and proliferation," Rice said.
The next and final phase of the agreement with North Korea calls for Pyonyang to hand over all nuclear material and weaponry in exchange for diplomatic ties with the US and Japan, and a formal peace pact.
In return, Washington has eased some trade sanctions and moved towards taking the North off its list of state sponsors of "terrorism".
The US helped fund the demolition of the cooling tower with a $2.5m contribution.
"There is still much to be done but it's a good step when the North Koreans comply with their obligations," Gordon Johndroe, the US national security council spokesman said.
The Yongbyon facility produced the plutonium for North Korea's nuclear weapons programme which culminated in a nuclear test in October 2006.