Geelani vowed to continue the struggle until the territory is merged with Pakistan.
"Today I vow before you that the ongoing struggle will be taken to its logical conclusion," 73-year-old Geelani told an estimated 35,000 supporters at the rally in the summer capital, Srinagar.
Geelani supports the merger of Indian-controlled Kashmir with Pakistan.
Nuclear-armed rivals India and Pakistan each hold Kashmir in part but claim it in full. They have fought two of their three wars over the region.
"We will achieve our goal if we strictly follow Islam," said Geelani, who heads a faction of the region's main separatist alliance, the All Parties Hurriyat Conference.
Sunday's rally was organised by Geelani's political party, Tehreek-e-Hurriyat (Movement for Freedom).
Geelani repeated his opposition to autonomy for the region and to the conversion of the de facto border dividing Kashmir between the two neighbours into a permanent border.
India and Pakistan have been
engaged in a peace process
"The only solution to resolve the dispute is to provide right of self-determination to the people of Kashmir," he said.
"The implementation of UN Security Council resolutions is the best and right way to resolve the dispute," he said of five-decade-old resolutions that would give Kashmiris a choice between India and Pakistan through a UN-held referendum.
India says the resolutions are obselete while Pakistan says it will not insist on a vote if India shows flexibility in resolving the dispute.
The two neighbours have been engaged in a 20-month-old peace process. They have already resumed a bus service between the two zones of Kashmir.
On Saturday they took important steps towards reducing the risk of an accidental nuclear war by agreeing to set up a telephone hotline and to notify each other before testing ballistic missiles.
Geelani's supporters travelled from across the Muslim-majority Kashmir valley waving green and blue party flags. The rally was one of the biggest held by a separatist leader in recent times.