One witness allowed into the court said a first batch of 10 defendants appeared in the dock on Monday.
A total of 31 are expected to stand trial.
The foreign press has been barred from attending the court proceedings.
All 10 defendants pleaded not guilty to charges of "belonging to a banned secret organisation" and of "attempting to overthrow the regime by armed force".
They said they "sought to disseminate and promote religious ideas … among students".
A further 10 defendants are to appear in court on Tuesday with the remaining 11 being brought before the bench on Wednesday when a verdict is expected, court officials said.
Unconfirmed reports said the group was suspected of planning attacks on the Muscat festival, a month-long trade and cultural event spanning part of January and February, as well as commercial centres and oil installations.
The families of each of the accused were allowed to send a representative to attend the trial.
News of the arrests in January prompted the authorities to confirm that a group of alleged Islamists had been detained and that its members were seeking "to set up an organisation aiming to harm national security".
But the authorities played down any perceived threat of terrorist attacks of the sort that have shaken Saudi Arabia, Kuwait and part of Yemen.
"The arrests have nothing to do with terrorism or foreign parties," said deputy prime minister for cabinet affairs, Fahd bin Mahmud Al Said.
Oman, which produced 800,000 barrels of oil a day in 2003, has around 2.3 million people in a territory half the size of France, and occupies a strategic spot opposite the Strait of Hormuz at the entrance to the Gulf.