Speaking at Singapore's International Institute of Strategic Studies on Saturday, Rumsfeld said the Pentagon's annual assessment of China's military capabilities showed Beijing was spending more than its leaders acknowledge.
He added that its government was expanding missile capabilities and developing advanced military technology.
China has the world's third-largest military budget, Rumsfeld said, behind the US and Russia.
"One might be concerned that this build-up is putting the delicate military balance in the region at risk - especially, but not only, with respect to Taiwan," he added.
"Since no nation threatens China, one wonders: Why this growing investment?"
The US spends over $400 billion a year on its military. China's expenditure is estimated at $55 billion to $85 billion.
However, China's Ministry of Foreign Affairs spokesman Kong Quan has said that the separatist policies in Taiwan pose a threat to peace and stability in the region and that Washington was exacerbating the problem.
"China has requested the US to faithfully honour its commitments to China on the Taiwan issue.
"Stop sending wrong signals to the pro-independence forces in Taiwan … avoid causing new damage to the peace and stability across the Taiwan Straits and Sino-US relations".
China has repeatedly called on the US to stop selling weapons to Taiwan and has said it would attack Taiwan if the island tries to declare independence.
North Korea negotiations
Rumsfeld's speech prodded Beijing to use its influence over North Korea and help restart six-nation talks over Pyongyang's nuclear weapons programmes.
Those talks have been stalled for a year, and US officials worry Pyongyang - which declared itself a nuclear power earlier this year - may be preparing to test a nuclear bomb.
Rumsfeld's speech was the strongest criticism levelled at China since he took charge of the Pentagon in 2001 and comes days after a decision to send 15 F-117A Nighthawk stealth fighters to South Korea for what the Pentagon called a long-planned training mission.
The previous low point was in April 2001, when a US spy plane and a Chinese fighter jet collided in mid-air, forcing the American plane to land in China. The crew of the Navy plane was released 11 days later.