Hamas also wants more passports, issued in the West Bank, for its Gazan officials and supporters.
Azzam al-Ahmed, head of Fatah’s parliamentary bloc, said on Sunday that mediation efforts to convince Hamas to allow Gaza-based delegates to attend had failed.
Mahmoud Abbas, the Palestinian president and leader of Fatah, had reportedly told Hamas through Arab mediators that if it agreed to let the Gazan delegates out, he would free 400 Hamas detainees - half before the congress and the rest during.
But Hamas said the onus was on Fatah to act first.
"We are willing to allow our brothers, members of Fatah’s sixth conference who are in Gaza, to travel, but our detainees held in the West Bank must be released first," Usama Hamadan, a representative of Hamas based in Lebanon, told Al Jazeera.
"If Mr Mahmoud Abbas's demand is necessary, our demand regarding the detainees is more than necessary."
The issue has cast a shadow over the conference, with some in the movement questioning how profitable the talks will be without the Gaza-based members.
Sultan Abu al-Ainain, a representative of the Fatah movement, also based in Lebanon, condemned the decision.
"Hamas had a chance to return to the good originality of the Palestinian people, but it had not taken this chance," he told Al Jazeera.
"It [Hamas] has very well proved that it is the other side of the enemy of the Palestinian people."
But Fatah officials insisted that the conference, Fatah's first convention in 20 years, would go ahead without the Gaza delegates.
Failure or cancellation of the congress could further weaken the already poor standing of Abbas and would be a blow to the US administration's push for peace.
In the 20 years since the last convention, peace hopes for the Middle East have fluctuated.
The conference plans to adopt a re-write of Fatah's 1989 convention, which called for "armed action" against Israel.
The new one firmly commits the Palestinians to peace talks, although it still mentions "armed struggle" as a theoretical right, said its author, Nabil Shaath, a veteran Fatah leader.
"We are dedicated to the peace process, we are dedicated to negotiations, provided we are not taken for a ride"
Fatah central committee member
He said the Fatah programme is setting 20 rules for its peace negotiators.
For example, it states that negotiations cannot be held as long as Israel expands Jewish settlements.
Abbas has said he will not resume talks without such a freeze, and such a clause could strengthen him should he come under international pressure to bend on the issue.
"We are dedicated to the peace process, we are dedicated to negotiations, provided we are not taken for a ride," Shaath said.
Fatah delegates from Syria, Lebanon and Jordan have already begun arriving for the Bethlehem congress with the consent of Israel, which also agreed to allow Gazan delegates to travel to the West Bank.
Israel has otherwise kept Gaza under a punishing blockade.