Politicians and journalists were ordered out of Stormont parliamentary building as the fire alarm sounded - and two security guards pinned Stone by both arms to the main doorway.
He was later wrestled outside, into pouring rain and wind, as he shouted: "No surrender!"
Stone had abandoned the bag near the building's security checkpoint staff, who operate metal detectors and search bags.
He also appeared to have been spray-painting the entrance to Stormont with "Sinn Fein-IRA murderers" or "war criminals," but did not finish it.
Peter Hain, the Northern Ireland secretary, ordered an immediate probe into what he called a "very serious breach of security. ... The chaotic scenes at Stormont were deeply disturbing."
Minutes earlier, Ian Paisley, the Protestant leader, refused to accept a nomination as the future leader of Northern Ireland's power-sharing administration.
Paisley, whose Democratic Unionist Party is the largest in Northern Ireland, said he would work with Sinn Fein, the IRA-linked party that represents most Catholics, only when it begins to support the police force.
If that happened, Paisley said, he would accept the post.
"When Sinn Fein has fulfilled its obligations with regard to the police, the courts and the rule of law, then and only then can progress be made. There can and will be no movement until they face and sign up to their obligations," he told the assembly.
The prime ministers of Britain and Ireland, Tony Blair and Bertie Ahern, said Stone's threat illustrated why rival British Protestant and Irish Catholic politicians should compromise and form a stable coalition as the Good Friday peace agreement intended.