Demonstrators wearing red t-shirts, hats and headbands gathered outside Chen Shui-bian's office as he presided over the annual national day celebrations.
Organisers, led by Chen's former ally Shih Ming-the, said that more than 1.5 million people had turned out for the protest. No police figures were immediately available.
"Chen Shui-bian and the ruling DPP as well as our government have all turned a cold shoulder to us. Chen must respond to the demands of the people. He must make a decision," Shih told supporters.
About 15,000 police and security officers were on duty to keep order, and barbed wire barricades prevented protesters from approaching the presidential complex where 50,000 people were attending the celebrations.
Opposition politicians from the Kuomintang (KMT) party who were invited to join Chen on stage as he gave his address shouted "Down with A-Bian" - the president's nickname - and gave him the thumbs-down.
Scuffles broke out between members of Chen's Democratic People's Party and the KMT after they unfurled red banners, accusing Chen of corruption and urging him to resign.
Chen Shui-bian has been accused
of misusing state funds
Chen used his 12-minute speech to call for national unity.
"Taiwan is a free country where freedom of speech is protected by the constitution," Chen said.
"We are allowed to have opposing views but we must not destroy national unity ... we must not divide the country," he told the audience.
Misuse of funds
The president is under pressure after he was questioned over the alleged misuse of funds intended for state affairs. He has denied any wrongdoing and pledged to stay in office until his second and final term ends in May 2008.
Prosecutors said last week that Wu Shu-chen, Chen's wife, had received and spent 300,000 Taiwan dollars ($9,090) worth of department store vouchers but cleared her of accepting the gifts in exchange for favours due to "lack of evidence".
Chao Chien-ming, Chen's son-in-law, has been indicted on insider trading charges.
On Friday, Taiwan's parliament is scheduled to vote on a second motion against Chen, but observers have said it is unlikely to pass.
The two opposition parties only hold 112 seats and the vote needs the support of 148 MPs.
In June, Chen survived an unprecedented parliamentary vote to force him out. If passed, it would have led to a national referendum on his future.