"No charge has been framed against them. Investigations are underway as to whether they have any links with extremist groups," a second Pakistani official said.
Officials said three Pakistanis had also been detained, one of whom was believed to have been linked to a 2007 suicide bomb attack on an air force bus outside an air base in Sargodha, in which eight people were killed.
"Some mobile phone SIMs and computer disks have also been recovered," a US official said on Thursday.
Pakistani news reports said the suspects were being investigated for links with the banned Jaish-e-Mohammad group, which has links with al-Qaeda and the Taliban.
It is one of several factions with roots in Punjab province that have for years been fighting Indian forces in the disputed region of Kashmir.
Jaish-e-Mohammad was suspected of involvement in several high-profile attacks including the murder of Daniel Pearl, a US Wall Street Journal reporter, in 2002, and an assassination attempt on Pervez Musharraf, the former Pakistani president.
Officials said that one of the Americans is of Egyptian origin, one is of Yemeni origin and another is of Eritrean origin.
The Council on American-Islamic Relations (CAIR), a US Muslim civil rights group, said it brought the case to the attention of US law-enforcement authorities this month after family members informed CAIR of the men's disappearance.
Nihad Awad, the national executive director of CAIR, told a news conference that an 11-minute video was left behind that appeared to be a "farewell" from the men.
The video did not state what the men planned, but that it mentioned world conflicts and featured verses of the Quran, he said.
News of the arrests came as a US national with Pakistani roots, accused of scouting targets for the 2008 Mumbai attacks in India, pleaded not guilty in a Chicago court on Wednesday at his first hearing since his October arrest.
David Headley, 49, is being investigated in the US for his alleged ties to Lashkar-e-Taiba, a Punjab-based group blamed for the two-day rampage in Mumbai, in which 166 people were killed.