Police say they found a notebook in von Brunn's vehicle with a handwritten note saying: "You want my weapons - this is how you'll get them. The Holocaust is a lie. Obama was created by Jews."
'White supremacist' views
Von Brunn, walked in to the museum holding a rifle at his side and began shooting, police said.
A court affidavit describes the shooting at the museum in detail and says it was captured on security video.
Witnesses said museum visitors ducked for cover and screamed as shots were fired.
The suspect is an alleged Holocaust denier who is believed to run a website detailing his white supremacist views and who once tried to kidnap members of the US Federal Reserve board.
Von Brunn, who was thwarted in that incident when a guard captured him outside a board meeting carrying a bag stuffed with weapons, was sentenced in 1983 to prison and released in 1989.
The US homeland security department called Wednesday's shooting a criminal incident but in a joint assessment with the FBI issued on Wednesday, said it did not appear to have a connection to "terrorism".