The march, organised by the opposition Democratic Progressive Party, is expected to be followed by more protests this week, including at the airport when Chen Yunlin, Beijing's senior negotiator with Taiwan, arrives on Monday.
Chen will meet P.K. Chiang, his Taiwanese counterpart, to discuss the proposed Economic Cooperation Framework Agreement (ECFA), a pact aimed at cutting import tariffs and opening the banking sector that should be signed next year.
"Many people in the crowd are business people who are concerned that their businesses will hurt if the free trade talks go ahead," Al Jazeera's Steve Chao, reporting from Taichung, said.
"It's all very clear [China] wants to use the economy as a means to force us to unification"
"A lot of criticism here has been directed at Taiwan's president, Ma Ying-jeou. They feel he is moving too far, too fast with China and they are calling on him to rein back the momentum on warmer relations with China."
Hsu Wen, a 55-year-old businessman from the southern city of Kaohsiung, told The Associated Press news agency: "It's all very clear [China] wants to use the economy as a means to force us to unification."
Some protesters feared the ECFA would lead to a flood of competing goods from China, while other called for the talks to be more transparent.
"Ma Ying-jeou, our president, wants to sign ECFA but hasn't received public approval for it," Charles Lee, president of an environmental group in southern Taiwan, said.
"We're worried he will sell us out."
The Ma administration has promised the pact will lift growth and create employment.
Also among the thousands of people on the streets of Taichung were supporters of a formal declaration of independence from China. Some waved banners advocating "one side, one country".
China has claimed sovereignty over Taiwan since 1949, when the communist forces of Mao Zedong won the Chinese civil war and Chiang Kai-shek's Nationalists fled to the island.
Beijing has previously vowed to bring Taiwan under its rule, by force if necessary.
Since assuming office, Ma has eased tensions to their lowest level in 60 years, turning his back on his predecessor Chen Shui-bian's pro-independence policies while pushing business and trade links.
Regular air and sea links have been established across the strait separating Taiwan from China and restrictions on Chinese investment in Taiwan have been ended.