Johnston's captors had declared themselves to be the Army of Islam, a previously little known group with links to the powerful Dughmush clan.
Johnston arrived in Jerusalem after being freed early on Wednesday from what he described as "an appalling experience, saying it was "an amazing thing to be free".
Al Jazeera's Nour Odeh said Johnston had been freed by his Army of Islam captors but looked frail and had lost a lot of weight in his nearly four months of captivity.
She said his release came after lengthy talks and strong pressure by Hamas on the Army of Islam group.
"It is just the most fantastic thing to be free," Johnston told BBC from Gaza, adding that it was "at times quite terrifying" not knowing when it was going to end.
"I dreamt many times of being free and always woke up back in that room ... It's an amazing thing to be free," Johnston said at the home of Ismail Haniya, the former Palestinian prime minister and local Hamas leader who had brokered his release.
Johnston called his 16 weeks in captivity the worst of his life and likened his experience to being "buried alive".
But he said he had followed events on a radio during most of his time as a hostage and thanked people around the world for what he called a psychological boost from their support.
"It's just the most fantastic thing, to be free. It was an appalling experience, as you can imagine, 16 weeks kidnapped.
I am hugely grateful to all the people - an amazing number of people - that worked on the Palestinian side, the British government, the BBC from top to bottom, and the huge amount of support from BBC listeners.
I think I'm okay. It was an extraordinary level of stress and psychological pressure for a long, long time, and obviously difficult to keep your mind in the right place ... a constant battle.
I do feel I probably got through it as well as I could have. I probably won't know for a while, but I feel as well as I could I think."
Johnston praised Hamas for winning his freedom. "If it hadn't been for that real serious Hamas pressure, that commitment to tidying up Gaza's many, many security problems, then I might have been in that room for a lot longer," he told a news conference.
Asked if he would return to Gaza, Johnston said: "I spent three years covering Gaza as a correspondent and I spent four months in solitary confinement there, and I feel - enough already with Gaza.
"You know, maybe I'll go back when it's a member of the EU," he said.
Johnston was the only Western correspondent working full-time in the coastal enclave when he went missing on March 12 after his car was found abandoned.
Late on Wednesday, Johnston won an Amnesty International award for human rights reporting and raising public awareness with his radio work 'Dispatches from Gaza'.
The annual award was announced on BBC television and Johnston had been chosen as the winner before his kidnapping.
Johnston's parents said they were relieved to hear that their son had been freed.
"We've seen him on the box, and it's just incredible. It's been a long 114 days," Graham Johnston, Alan's father, said.
The BBC said in a statement it was "extremely relieved".
The Army of Islam had issued web videos showing Johnston and seeking the release of Islamists held prisoner by Britain and other states.
Johnston was pictured wearing a suicide vest with the warning that he would die if Hamas officials acted on their threat to free him by force from the clan's stronghold.
Hamas had increased pressure on the hostage-takers to relent and had surrounded the group's neighbourhood late on Tuesday.
Hamas takes credit
Khaled Meshaal, the group's exiled leader, told Reuters from Damascus: "The efforts by Hamas have produced the freedom of Alan Johnston."
Hamas fighters surrounded the area of Gaza City that is the base of the Dughmush clan, one of whose leaders, officials say, is also a leading figure in the Army of Islam.
It also exchanged prisoners with the group in recent days during negotiations to free Johnston.
Hamas seized control of Gaza last month after its forces defeated those of Mahmoud Abbas, the Western-backed Palestinian president.
Abbas responded by dissolving the unity government he had run with Hamas and appointing an emergency government that has attempted to isolate the movement.