The Dongria Kondh tribe, a group of about 8,000, believe their god, Niyam Raja, lives in the remote hills where the proposed mine was to be located.
The area is home to deer, antelope, elephants and the rare Golden Gecko lizard. Campaigners were worried that a large mine there would led to deforestation and environmental contamination.
The controversy has been compared to the blockbuster movie Avatar, where a group of tribal people fight a ruthless company intent on taking their land in order to build mines that destroy traditional lifestyles and the environment.
"The Dongria's campaign became a litmus test of whether a small marginalised tribe could stand up to a multinational company," Jo Woodman, a campaigner with the advocacy group Survival international, said in a statement.
Vedanta could not be reached for immediate comment, but the firm, owned by billionaire Indian business man Anil Agarwal, had previously said the project would bring jobs and development to a poor region.
The decision is a major blow to the firm. Currently, it operates an aluminum refinery in Orissa, but it must import costly bauxite to make the product. The inability to produce its own bauxite could seriously hurt the company's profitability.
Some local officials in Orissa were also upset. Raghuanth Mohanty, Orissa state industry and mines minister, said the decision was "extremely unfortunate".