Ken Ogeto, Erlinder's lawyer, said his client understood that he would be allowed to leave the country as he had been given an unconditional release.
"We have been busy with the question of bail for the last two weeks. We lost in law court but today the high court managed to guarantee his bail.
"We are very happy about it, the bail was given without any condition, and the only condition actually is to leave an address here, which means that he can travel back to the USA," Ogeto said.
His lawyer said: "The prosecution was given the authorisation to proceed without investigations. We don't know how long we can wait but our client is willing to come back anytime they need him with absolutely no problem".
The case has strained relations between Rwanda and the ICTR, which was set up to try those responsible for the most serious crimes committed during Rwanda's 1994 genocide.
Erlinder filed a lawsuit earlier this year accusing Paul Kagame, the Rwandan president, of ordering the killings that sparked the genocide. He is the first foreigner to be tried under Rwanda's 2003 anti-genocide legislation.
The ICTR had earlier said Erlinder should not have been held as he had immunity and Rwandan prosecutors used a statement made in a case at the international court as evidence in the case against him in Rwanda.
A defence counsel at the international court, Erlinder went to Rwanda to defend Victoire Ingabire, an outspoken presidential candidate who was arrested on genocide denial charges in April and later released on bail.
The US lawyer has been in hospital four times since his arrest complaining of heart problems and panic attacks.