India and Pakistan have fought three wars since their independence in 1947.
Tension surged again in 2008 when attackers alleged by India to be members of Lashkar-e-Taiba, attacked the financial hub of Mumbai, killing 166 people.
Gates praised India's response to the attacks, but said: "I think it's not unreasonable to assume India's patience would be limited were there to be further attacks."
New Delhi suspects the Pakistani intelligence service of supporting armed groups that target India and has consistently called on Islamabad to crack down on fighters operating on its soil.
Gates described India as a vital partner in the struggle against extremist threats and said that he had discussed how to bolster US-India military co-operation.
He also lauded India's "extraordinary" financial aid to Afghanistan, while acknowledging the tension this created as both Islamabad and New Delhi vie for influence in Kabul.
"There are real suspicions both in India and Pakistan about what the other is doing in Afghanistan," he said.
"So I think each country focusing its efforts on development, on humanitarian assistance, perhaps in some limited areas of training, but with full transparency for each other, would help allay these suspicions and frankly create opportunities."
Gates, after a cultural trip to visit the Taj Mahal on Wednesday afternoon, was to head to Pakistan on Thursday.