For hours, the local Abbottabad police kept us a fair distance away from the house that Osama bin Laden was killed in.

Every time we tried to set up our camera to film the imposing three-storey white house from a distance, angry officers rushed over to stop us.

Then, in the early afternoon - on whose orders we'll never know - the police allowed the media to get close to the sprawling compound.

Correspondents and camera operators rushed to the tall green gates to start filming – jostling each other for the best position.

Behind them, a stream of neighbours that turned into a flood of curious onlookers most more interested in the crowd of journalists than in the house the world's most wanted fugitive is believed to have lived for around five years.

During the height of the frenzy, a moment of macabre humour, when a bin Laden look-alike came to the area.

Wearing a long scraggly beard and the white headgear, so synonymous with the al-Qaeda leader, the imposter was greeted with cheers and jeers. 

Some of the crowd even started chanting "We are all Osama!" then broke into peals of laughter.

Pashtuns from Peshawar

But the circuslike atmosphere couldn't overshadow the picture that is starting to form of what life was like behind those walls.

Police told us behind the imposing four metre high walls, barbed wire and security cameras, a large vegetable garden, cows, chicken and stores of food - clear signs the compound was relatively self-sufficient for those who lived inside.

Neighbours say the only people they ever really had contact with were two brothers: Pashtun's, they said, from Peshawar.

When asked about the high walls, they said it was because their wives observed purdah. 

It should be noted that it's not uncommon to see this sort of construction in Pakistan's tribal areas.   

Khourshid Bibi, an elderly woman who lives down the road from the compound, told us one of those men was called Nadeem.

"I didn't really know him too well. We would only ever say hello to each other. Just like you would with anyone you recognise, but we didn’t have any kind of relationship with him. I once saw him with a child, who was 7 or 8, they were Pathan’s speaking pushto."

She said the man she knew as Nadeem rarely interacted with anyone but did attend funerals and gave charity to the poor. She, however, added she found the compound he lived in strange.

"The house was so secure, just like when you shut something in a box and it doesn’t ever come out. We rarely ever saw anything or anyone."

According to reports, 'Nadeem' could very likely have been bin Laden's courier, the man trusted to take his video and audio messages to the outside world.  

US officials described him as a Pakistani brought up in Kuwait.

Nightmare begins

And while the manhunt for the world's most wanted man has come to an end, for the family who live just opposite, it is the beginning of a nightmare.

Mohammed Qasim told us his father was arrested after the early morning operation carried about by US Navy seals in which an unarmed Osama bin Laden was shot twice and killed.

"At around midnight, my family heard helicopters," he said.  "We didn't leave the house because there was so much bombing and firing outside ... It was only after whoever those men were had left that the Pakistani army showed up. An army official called my father out of our house and ever since then we haven't seen or heard from him."

Local police confirmed that Mohammed Qasim's father was taken into custody but wouldn't say why or when he would be back.

Qasim says he doesn't know why his father was taken, saying the family had nothing to do with their closest neighbours, one now revealed to have been the world's most wanted fugitive.

As well as bin Laden, five other people - including bin Laden's son Khalid - died in the 40-minute operation carried out by US forces.

A number of others, including women and children, are reported to have survived the raid and are under Pakistani custody in nearby Islamabad.

The house itself remains under close guard, but there are some suggestions it could be pulled down so that it doesn't become a shrine for al-Qaeda sympathisers. 

But pointed questions for Pakistan remain. How could the world's most recognisable fugitive have lived in this Abbottabad community’s most distinctive house, a mere kilometre away from the nation's top military academy, without their knowledge?

How, some ask, could Osama bin Laden have hid from them in plain sight?