"If Russia does not step back from its aggressive posture and actions in Georgia, the US-Russian relationship could be adversely affected for years to come," he said.
Gates also said that a US-Canadian-Russian military exercise due to take place on August 20 had been cancelled.
The defence secretary's warning came as there were unconfirmed reports that a large Russian convoy was moving towards Georgia's second city of K'utaisi in the west of the country.
Earlier on Thursday, Russia warned Washington against encouraging Georgia's leaders to take steps that might lead to a repeat of the "tragic scenario" of recent days.
"We expect the United States to adopt a responsible approach," Russia's foreign ministry said in a statement.
"It is especially important to refrain from any steps that could directly or indirectly be taken by the Georgian leadership as encouragement of its revanchist ambitions and prompt a repeat of the tragic scenario."
Separately, Sergey Lavrov, the Russian foreign minister, issued a blunt message to Georgia and the world that appeared to challenge a demand a day earlier by George Bush, the US president, that Russia must respect the sovereignty and territorial integrity of Georgia.
"One can forget about any talk about Georgia's territorial integrity because, I believe, it is impossible to persuade South Ossetia and Abkhazia to agree with the logic that they can be forced back into the Georgian state."
The White House said on Thursday that the US position was unchanged and dismissed Lavrov's remark as bluster.
Jonah Hull, Al Jazeera's correspondent in Tbilisi, the Georgian capital, said that the standoff between Georgian and Russian troops around the Georgian city of Gori could be read as a message to Washington over its involvement in the conflict.
|The concessions allow Russian troops to patrol 10km out of South Ossetia [AFP]
The Russians were deeply annoyed at the blatant American involvement, he said, and the standoff around Gori may be simply to show just how annoyed they were.
Mikheil Saakashvili, the Georgian president, called on the international community to "think very carefully about what is going on here. We need to stop everything that can be stopped now".
He accused Russia of a "deliberate policy of ethnic cleansing".
"I can prove it with the international organisations already bringing testimony to what I'm saying," he said at a briefing for foreign media.
Bush earlier re-affirmed that the US remained fully committed to "a sovereign, free Georgia and its territorial integrity".
But aside from the rhetoric, the US has so far sent only humanitarian relief to Georgia.
"Right now, the only people we will have on the ground are those that are required to deliver the human mission. Our priority is to save lives and alleviate suffering," Gates said.
A second C-17 US military cargo plane carrying humanitarian assistance for Georgia arrived in the capital, Tbilisi, on Thursday morning.
But the Russians are suspicious of the cargo and they are upset that Condoleezza Rice, the US secretary of state, is visiting Tbilisi but not Moscow.
Rice's trip to Tbilisi on Friday, however, appears to be to press Saakashvili to sign a formal EU-brokered ceasefire agreement with some new concessions to Moscow.
The new document would allow Russian peacekeepers who were in the disputed South Ossetia region before the fighting broke out a week ago to stay, and they would now be permitted to patrol in a strip up to 10km outside the area, US officials said.
But that allowance would be temporary, and details were still to be worked out, the officials said.
Nicolas Sarkozy, the French president, who met Rice in Paris for talks on Thursday, said the document was "intended to consolidate the ceasefire".
Sarkozy, who has been leading Western diplomatic efforts to end the conflict between Russia and Georgia over the breakaway provinces of South Ossetia and Abkhazia, said the situation on the ground was "improving".
"If tomorrow, President Saakashvili signs these documents, then the withdrawal of the Russian troops can start," he said.
But Rice said Russian military actions on the ground indicated the country had not respected the truce and urged Moscow to do so.
Ivan Safranchuk of the Moscow State Institute for International Relations told Al Jazeera that international powers were leaving themselves open to charges of a double standard.
"It seems that America and Nato can make a humanitarian intervention when they think it is right, but Russia cannot make an intervention when it thinks it is right.
"In Russian eyes the situation already seems like double standards," he said.
Nile Gardiner, a former foreign policy adviser to Margaret Thatcher, the former British prime minister, rejected any comparisons between the US and Russia.
"There's a huge difference between the United States and a dictatorial regime like Russia. After all the Russians have been the aggressors against Georgia."