Jem signed a deal with the Sudanese government after talks in the Qatari capital, Doha, last month, under which both sides agreed to undertake "good faith" measures.
"There was supposed to be a conference [in Doha] after three weeks but we will not go," Ibrahim said.
He said the government must allow the expelled aid agencies back into Darfur and free Jem prisoners before talks could resume.
Ahmed Hussein Adam, a Jem spokesman, refused to rule out a renewed military conflict with the Sudanese military and allied Janjaweed militias.
"The government doesn't want peace, the government wants to impose a security solution, a military solution on our people," he said.
"We are going to do everything to protect our people and we are going to do everything to stop Bashir killing our people again."
Aid officials said on Thursday that there was a risk of fresh disease outbreaks in south Darfur's Kalma and Kass camps after residents refused to let state-backed aid agencies come in to replace the expelled groups.
Hussein Abu Sharati, who says he represents Darfuri refugees in 158 camps, said Kalma residents had met and voted to refuse all aid from the Sudanese groups that al-Bashir said would fill the gap left by the international organisations.
"They don't see these groups as aid organisations, they see them as tools of the government," he said.
The US embassy in Khartoum said it was "deeply concerned" by the situation in Zamzam camp in north Darfur, where the expulsions have coincided with the arrival of about 36,000 people fleeing recent fighting.
United Nations officials say that about 300,000 people have died and more than 2.2 million people have been uprooted by fighting in Darfur, which began when the government decided to put down an uprising by ethnic rebels.
Khartoum disputes the UN figures, saying that only about 10,000 people have been killed.