It became notorious as the birthplace and headquarters of the Red Command, the biggest drug faction in the country, and was the scene of one of the greatest escapes of all time when the faction's leader was taken off the roof by helicopter.
In the 1960s and 70s it housed a mishmash of writers and revolutionaries during the military dictatorship.
Now, all that remains of the Instituto Penal Candido Mendes is a facade after its demolition in 1994.
A handful of low-risk inmates continued with restrictions on their movement in the remote corner of the island, Two Rivers.
Ten years later, only one remains. Julho de Almeida, 74, is the last prisoner on Big Island.
Home from home
After spending 26 years inside the famous jail, Julho still lives within sight of the place.
He walks with a stick that has its own story. "This is a type of wood that bends but doesn't break," he says.
The notorious prison is now
empty and derelict
"The guards used it to hit prisoners who stepped out of line. It was one of those weapons talked about in the prison years ago. Today I use it to walk around," he says.
"I can't be here thinking about the prison. I, for example, have to forget. I do forget that there was ever a prison here. I say that, but no one believes it!" he laughs.
He refuses to reveal the reason behind his long sentence, and no one else on the island either knows or is willing to tell.
"It's not that I don't want to talk about it; but it's never good to record the past and so I have never said and I am not going to say, because it is sad to recall the past. I don't think you should. There are people who like to tell, but not me," says Julho.
Walking around the ruins, Julho points out what he remembers - where there was a bakery, where they looked after 300 head of cattle, where the guards slept and where there was the most feared place, the isolation cell.
A cell window is now overgrown
"This cell was a place where one person stayed in isolation within four walls with just a little bathroom to wash and everything else. And they lived inside. We had two cells like this. One was where Hugo Ferras stayed. He killed and he was sent there by the very prisoners, because he killed inside the prison. Inmates judged others.
"He was in isolation for six years. Just eating and sleeping and nothing else. He was given food through a hole. His flesh wasn't resistant, without oxygen and sun. Eventually Pope John Paul II answered his appeal and he was freed to a common cell. With time he recovered. But his skin was very weak after six years in isolation," remembers Julho.
It was 30 years ago that the Red Command was created, the result of a marriage between the organisational skills of the revolutionaries and the criminal minds of the rest according to legend.
Today, the faction is a virtual parallel power in Rio de Janeiro and controls most of the territories of the giant favelas, or slums, in an undeclared civil war.
"It was created here in 1974. There were a lot of fights between them and a rival gang because of drugs. There began a little part of the movement outside and then a war started between them inside. When they were created, Comando Vermelho began killing people, two or three a day, because of the movement," remembers Julho.
The natural environment makes
the island a tourist retreat
"When they invited me to be part of it, I didn't want it. I talked to the leaders, because you weren't obliged. There was a certain camaraderie in relation to drugs when you left the prison. You would leave with a job already in hand, some drug dealing point or whatever. I said I was here recuperating and I didn't want to be part of it. We had a system where, if you saw things, you kept your mouth shut," he said.
On a dramatic New Year's Eve in 1985, the Command's leader, Escadinha, was taken out in a hijacked helicopter. He lived free until he was shot dead by police in Rio last year.
But the island has not become a forgotten backwater - in fact, the opposite. Because, for more than 100 years nothing and no one had been allowed on to Ilha Grande, as it is called in Portuguese - outside the movements of security and prisoners - the island's ecology had been left virtually untouched.
Ten years on, there is a thriving eco-tourism industry and the number of pousadas (hostels) exploded to 70 in just two years.
The laid-back pace of island life is obvious to anyone arriving. There are no cars - just one police van and a fire truck. More than 100 white sand beaches are accessible by boat or by trails through the pristine rainforest - almost all without so much as a shack in terms of development.
For those who were born on the island, like 36-year-old waitress Adriana de Brito, the changes have been extraordinary.
"When the prison ended, there was the explosion in tourism. There used to be just five simple pousadas. Now there are more than 70. For us who live free on the island now we can't do this, can't do that. There's lots of new laws.
Waitress Adriana de Brito was
born on the island
"We were used to having parties, but where we used to dance is now another pousada. It was a small community. The people who are arriving now and investing have more power than us islanders," she says.
Robberies and drugs were non-existent, the internet was another universe and life was simple, she says.
"For the older people, it's too much information one hour to the next, they can't assimilate all this. To live here was better, now it's totally different. For me, for us islanders, it was better when we had the prison," says Adriana.
The island boasts more than 100
white sand beaches
But Julho believes he is living in paradise. "This prison was much-talked about because it was prohibited to come here and see it, so people didn't really know what kind of a place this was.
"When it was demolished and people were free to come and see and, well, today people admire the place, a place so beautiful, so tranquil. If people had known this years ago, the prison would have been destroyed much quicker," he says jokingly.