He has also restricted trade with Colombia as the dispute between the two nations has grown increasingly bitter.
Silva did not specifically mention Venezuela in his interview with El Tiempo, but spoke of the "growing risk" of a foreign power taking action against Colombia.
"In Colombia we have concentrated on the internal threat. But the risk is growing because what has been clearly and directly presented, is an eventual action against Colombia from outside," Silva told the newspaper.
"Colombia was not used to thinking about this eventuality in its foreign policy and defence strategy. Unfortunately now we have to put this variable on the map. There is a risk of a foreign aggression."
The new Colombian military base is to be built on the Guajira peninsula, near the city of Nazaret, and is expected to house about 1,000 troops, Silva said as he announced the development on Saturday.
"It is a strategic point from a defence point of view," he said.
Meanwhile, General Oscar Gonzalez, the commander of the army, said that six air battalions were being activated, including two on the border with Venezuela.
Relations between the two countries have previously looked as though they could spill over into conflict along the 2,200km border.
Colombia accused Venezuela of failing to deal with guerrillas hiding in its territory as the Farc and ELN Colombian rebel groups came under increasing pressure from a US-backed military offensive.
While Chavez ordered tanks to the Colombian border in 2008 after Colombia raided a suspected rebel base inside Ecuador.
Rafael Correa, Ecuador's president and an ally of Chavez, had condemned the raid as an aggression against his country, but tensions eased after the leaders met at a summit a week later.
The two Andean countries also almost went to war in 1987 in a dispute over a maritime border in the Caribbean Gulf.