The supermarket chain is the third largest in the country after Tesco and number two Asda, which is owned by the US giant, Wal-Mart.
J Sainsbury, meanwhile, is not Qatar's only current major investment in Britain.
The Gulf state is said to be a key investor in an expensive property development in central London.
Qatar's foreign minister is reported to have bought one of the apartment's for 100 million pounds, making it the most expensive flat in London.
Tim Attenborough, a financial analyst at Exane BNP Paribas, said he believed the Qataris did not aim to launch a bid for J Sainsbury.
Attenborough said he expected the group to use their stake as a lever to force it to release value from the property it owns.
"This increases the likelihood there will be a property-related deal," he said.
Alliance Bernstein, which held 254 million shares in the company, declined to comment on speculation it had sold its stake to the Qataris.
Robert Tchenguiz, a property invester and holder of a 5.07 per cent J Sainsbury stake, has already said he wants the supermarket chain to release more value to investors from its real estate assets.
J Sainsbury escaped a 10.1 billion pound bid by CVC Capital Partners this month pitched at 582 pence a share, following opposition from the company's founding family, which owns a stake of about 18 per cent.
CVC wanted J Sainsbury to sell some of its estimated 8 billion pounds of property assets in order to release funds to shareholders.