The soldiers gathered on the lawns of the Union Buildings, where central government is based, with some people attempting to climb the buildings' surrounding fence.
This followed a court refusal of the troops' application to march to demand improved pay.
"The military police and other police informed them they have to leave, they refused and then they tried to climb the fence," Ndhivuwo Mabhaya, a ministry spokesman, said.
"Our position is that the march was illegal. The union needs to take responsibility for the actions of their members. We are now consulting with our lawyers whether the union can be held liable for damage to property."
However, local television footage showed police firing into the crowd, forcing protesters against a fence, over which many had retreated. Police were seen to continue firing in the area, which contains embassies and hotels.
The protest ended when police warned of further force.
Sisulu added that any troops joining the protest would be put on unpaid leave, and that those immediately involved had been suspended.
She said that two soldiers had been arrested and handed to military police.
The South African National Defence Union was demanding a 30 per cent increase in pay. The union is not officially recongnised by the government and Sisulu called the demand "deliberatively provocative".
She added that there was now the potential that military bases were unprotected and that there was a lack of troops to support police if necessary.