Sin Chung-kai, Democratic party vice-chairman, said it was highly likely the party would lose seats in the 60-member legislature.
"I think the odds that we can not keep 20 seats are very high ... I would describe the situation as the pro-democracy camp's worst showing since there were elections," Sin said.
If the pro-democracy parties slip below 21 seats they will lose the ability to veto government legislation, which requires a two-thirds majority. They successfully vetoed a number of policies in 2005.
In Sunday's election, people voted to fill 30 legislative seats, with the rest chosen by members of special interest groups such as businesspeople, lawyers and accountants. The special groups lean towards supporting China.
Results are due in by Monday.
Analysts say Chinese nationalism among the city's 3.37 million voters has been growing due to the Olympic games.
In the 2004 elections, and against the background of strong anti-government sentiment, pan-democrats won about 60 per cent of the vote, giving them 25 seats.
Donald Tsang, the city's chief executive - who was elected by a small group of mainly pro-China figures last year - is slipping in the polls.
But this time round, the opposition has been unable to take advantage of the government's waning popularity.
Hong Kong was promised universal suffrage for both its legislature and chief executive when colonial power Britain handed the territory back to China in 1997, but no specific timetable was set.
The Democratic Alliance for the Betterment and Progress of Hong Kong (DAB) could benefit from the city's growing nationalism and from Beijing's announcement late last year that universal suffrage could be introduced by 2017.
But Michael DeGolyer, a politics professor at Hong Kong Baptist University, said his pre-election poll showed patriotism will not necessarily translate into more pro-Beijing votes.
"People are starting to treat their votes in the same way they deal with their investments," DeGolyer said.
"I do not know what the 'China factor' is. If the definition is 'support your country, vote for the DAB,' that is not the case any more."