The two soldiers were arrested during a raid on Tuesday on a house in a gated community in Carmen de Apicala, located southwest of the capital and near Colombia's sprawling Tolemaida air base, where US soldiers are stationed.
National Police chief General Jorge Daniel Castro said the two soldiers, whose names and ranks were not disclosed, were arrested at the house where a large weapons cache was discovered.
A US Embassy spokesman confirmed the arrests and said US diplomats are "working to ascertain the facts surrounding the case".
He refused to provide details, other than to say more information would be released as it became available.
But the Colombian attorney-general's office said the pair were being held by Colombian authorities near Carmen de Apicala.
Details on the type and quantity of arms found were not immediately available.
Castro said police in the village, 80km southwest of Bogota, stopped a suspicious man who immediately offered a bribe to be allowed to go free.
Under threat of arrest, the man led the officers to the nearby house where the arms stockpile was stashed.
The Colombian government is
fighting a major insurgecy
Shortly afterwards, the two US soldiers - apparently unaware of the police operation - entered the house but could not justify their presence.
"In the course of the investigation, two Americans arrived, they did not give a satisfactory explanation and were put at the disposal of the prosecutors' office," Castro said.
The incident marks the latest US embarrassment in the South American nation.
On 29 March, five US soldiers were arrested after 16kg of cocaine were found aboard a US military plane that flew to El Paso, Texas, from the Apiay air base east of Bogota.
One of the suspects has since been released, but the rest are being held in the US.
Colombian lawmakers called for their extradition to face trial in Colombia, but US Ambassador William Wood ruled out such a move, citing diplomatic immunity.
The illegal drugs business is
Colombia's major industry
The US has provided more than $3 billion in aid over the past four years to help Colombia battle leftist rebels and drug trafficking that fuels the 40-year-old insurgency.
Outlawed rightwing paramilitary groups are also in the fray, battling the rebels while committing massacres, trafficking drugs, and carrying out kidnappings and extortion.
Up to 800 US troops are permitted in Colombia, according to US law, to simultaneously train Colombian armed forces and provide logistical support.
Up to 600 Americans are also permitted in the country as US government contractors.