'Motivated by vengeance'
The AFP news agency quoted Meilleur as saying the suspect had confessed to the killings, which took place late on Thursday morning in a building 100 metres from the Palace of Justice in the centre of the capital.
"According to his own statement, he was present in the tribunal, he took out a weapon and he fired on the judge and her clerk," said Meilleur.
"He explained that he was motivated by vengeance against the judiciary in general and this justice of the peace in particular," who "decided in 2007 to evict him from the apartment he rented".
Meilleur said the investigation into the killings would "have to determine whether the man was mentally ill".
It was the first time a judge had been killed in a Belgian court, the justice minister said on Thursday.
The suspect is said to have stood up near the end of a court session and gunned down Isabelle Brandon, a magistrate, and her clerk Andre Bellemans, then fled from the courthouse on foot.
He was caught eight hours later in a nearby park after a brief exchange of gunfire with police.