Both the governing and opposition coalitions claim they have the right to rule Perak, which has been embroiled in a tense political and constitutional crisis since January.
The state has had no function government following the defections of several opposition legislators in January.
Police did not intervene, but stepped in to push back the opposition supporters when they approached the makeshift fence.
"This is utter rubbish. Elected representatives are not allowed to go inside," A Sivanesan, an opposition legislator, told the Associated Press.
"It is not in accordance with the law. The police [are] practicing double standards."
The crisis in Perak highlights the growing uncertainty in Malaysian politics after the ruling party suffered its worst ever election loss last year.
In the March 8 national polls, the ruling Barisan Nasional (BN) coalition lost control of four states including Perak, as well as its long-standing two-thirds majority in parliament, conceding a large number of seats to the opposition.
Last month opposition legislators protested after claiming the state sultan had acted unconstitutionally in appointing Zambry Abdul Kadir, a BN member, as chief minister without waiting for a vote of confidence in the state assembly.
Following the sultan's endorsement, V Sivakumar, the state parliament's speaker and a member of the opposition coalition, suspended the legislature.
He later called for opposition members to gather for an emergency session – a move seen as symbolic rather than having any real prospect of returning the opposition to power.
After being blocked from entering the state parliament the opposition legislators retreated to a side street where they held the emergency session in the open air.
The meeting unanimously passed a resolution of confidence in the ousted government led by Mohammad Nizar Jamaluddin.
"We are the rightful government ... Now the state has two chief ministers, one that is legal, one that is absolutely illegal," he said.
"The crisis must be solved ... through the dissolution of the state assembly."