The official said one of the men had been arrested because he had overstayed his visa.
A second man had already been awaiting a ruling from an immigration court hearing on whether he would be removed from the US.
"We can confirm that several search warrants have been executed in the northeast in connection with the Times Square bombing investigation," Gail Marcinkiewicz, an FBI spokeswoman, told the AFP news agency.
'No immediate threat'
Faisal Shazhad, a Pakistani-born US citizen, has already been charged with terrorism in connection with the botched May 1 attack, after he allegedly parked a car laden with a home-made bomb in New York's Times Square.
Authorities said Shahzad, 30, a former financial analyst who lived in Connecticut prior to the bombing attempt, has been co-operating with federal officials.
Dean Boyd, a justice department spokesman, said the searches were the result of evidence collected during a 13-day investigation.
Shahzad faces five terrorism-related charges. But he has still not appeared in court or been seen in public since his arrest on May 3 when he was hauled off a flight to Dubai.
In its statement on the latest arrests, the FBI said it could "provide no further details as the investigation is ongoing".
But it added: "This search is the product of evidence that has been gathered in the investigation subsequent to the attempted Times Square bombing and [does] not relate to any known immediate threat to the public or active plot against the United States."
The Pakistani Taliban claimed responsibility for the failed attempt to detonate a car bomb in the heart of New York City. If this claim is confirmed to be true, the attempt would be the group's first act in the United States.