"People are quite upset. There was trouble last night after the fire started. Firefighters were chased away by crowds of angry people," he said.
"[People] view this as an attack and we can expect serious unrest to follow."
There was no immediate confirmation of the cause of the blaze and police said that their investigations were continuing.
Buganda is one of Uganda's four historic kingdoms and Baganda the most populous ethnic grouping in the country.
Last September, riots in Kampala left at least 17 people dead after the government prevented Ronald Muwenda Mutebi, the current Buganda king, from visiting a district near the capital.
The government said the district had its own traditional ruler and it did not owe allegiance to the king.
The king holds a largely ceremonial position in Uganda, but wields considerable influence among his people, many of whom accuse the government of expropriating the kingdom's land.
Monarchies were restored in Uganda in 1993 after they were banned in the 1960s, but they are not allowed to participate in elective politics by campaigning or fielding candidates.
The kingdom's radio station - the Central Broadcasting Service (CBS) - remains banned after it was taken off air in the aftermath of the riots, its staff accused of inciting violence.