John Kerry has met Pakistani prime minister Yusuf Raza Gilani on a visit that the senior US senator described as "crucial".
Monday's visit to Islamabad was designed to ease distrust, given the tense relations between the two countries following the killing of Osama bin Laden by US forces on Pakistani soil.
President Asif Ali Zardari, General Ashfaq Parvez Kayani, the army chief, Rehman Malik, the interior minister, and Cameron Munter, the US ambassador in Pakistan, were also present at the meeting.
Kerry, a Democrat close to the Obama administration who is also chairman of the Senate Foreign Relations Committee, told Pakistan the US had "grave concerns" that the al-Qaeda leader was found in Pakistan, and that Congress was now scrutinising Washington's ties with its strategic ally.
But Kerry added that their strategic relationship was too important to let go.
He spoke at a news conference in Islamabad, also saying that both nations had to find a way to restore the trust between them. Kerry added that US-Pakistan ties were "too important to be stuck speculating".
"I emphasised to our Pakistani friends - and they are friends - that many in Congress are raising tough questions about our ongoing economic assistance to the government of Pakistan because of the events as they unfolded, and because of the presence of Osama bin Laden in Pakistan," Kerry said.
"The make or break is real. There are members of Congress who are not confident it (ties) can be patched back together."
Washington's already fragile ties with its ally took a beating after US special forces flew in from Afghanistan on a secret operation to find and kill bin Laden on May 2, nearly 10 years after he orchestrated the September 11 attacks on the US.
His discovery in the garrison town of Abbottabad, only 50km from the Pakistani capital, has deeply embarrassed Pakistan's military and spy agency and revived suspicion that Pakistan knew where he was and has been playing a double game.
Pakistan has rejected that as absurd, and its parliament has condemned the US raid as a violation of its sovereignty and called for a review of ties.
Kerry arrived in Pakistan on Sunday from Afghanistan where he told reporters the US wanted Pakistan to be a real ally in the fight against militancy.
In a televised statement, he said Pakistan was "recommitted to find more ways to work against the common threat of terrorism" and to increase co-operation on intelligence sharing and operations to "defeat the enemies that we face".
He added: "Secretary of State [Hillary] Clinton will soon announce plans to visit Pakistan to develop new trust".
"This relationship is not only about the threats that we face, it's about partnership with the people of Pakistan," Kerry said.
Kerry also announced on Monday that Pakistan had agreed to return the tail of a stealth US helicopter that American commandos had to destroy during the bin Laden raid because it malfunctioned.