In a report released on Wednesday, the charity said workers were paid poverty-level wages, faced poor working conditions and could be sacked if they took strike action.
The report said the brands paid millions to sponsor footballers performing at this summer's World Cup yet did not prevent sportswear workers from being paid as little as 60 cents an hour.
Oxfam produced a league table of 12 leading sportswear manufacturers based on their labour practices but said none met minimum requirements.
"They would all receive a red card if they were on the field," said the report's co-author, Tim Connor.
"In fact, if there was a world cup for companies that respect labour rights, the sports brands would not even get through the qualifying rounds."
Anton Supit, from the Indonesian Employers' Association, said that big sports companies had been tough on Indonesian suppliers and dismissed the abuse claims as "nonsense".
"They can ditch the suppliers if we cannot follow existing laws, especially on workers' rights," he said.
Oxfam rated Reebok as most worker-friendly of the manufacturers, whilst US-brand Fila came at the bottom of the league.
The report said one company, Tae Hwa, that had been producing shoes for Fila, suddenly closed, leaving workers unemployed and without severance pay.
"Since then FILA has declined to reveal its role in the factory's closure or to take responsibility to ensure Tae Hwa workers receive their legal entitlement to severance pay," the report said.
Before the Tae Hwa factory's closure, female workers complained that sexual harassment was common.
"In the sewing department sexual harassment happens all the time and every day. The managers will come near us and say things or look at us in a disturbing way," one woman told an Oxfam researcher.
Oxfam said workers at another factory inhaled rubber fumes during their shifts.
Workers at a factory used by Adidas were sacked after they participated in strike action, the report said.