Fleiss, who was arrested in 1995 for running a high-class prostitution ring for rich and famous clients, was testifying against Sizemore in Los Angeles, California, on Tuesday.
She said the actor had given her a black eye and a “fat lip”.
Sizemore has played roles in movies such as Saving Private Ryan and Black Hawk Down.
The film star is charged with 16 counts stemming from alleged incidents of domestic violence.
Fleiss, 37, hit international headlines when officials tried to force her to reveal her "black book" of prostitution clients, who included influential Hollywood figures and stars.
But before tearful Fleiss, took the stand, Sizemore's lawyer began by telling jurors that the former madam was a liar.
Wiping away her tears, Fleiss described a litany of attacks on her by Sizemore, whom she met in January 2001 before moving in with him in August of that year. The pair split up last year.
In one alleged attack in November of 2001, Sizemore beat Fleiss after she told “shock jock” Howard Stern during a national radio interview that she did not think much of his latest movie, she said.
"Tom (got) very mad at me because I said that I didn't like the movie (Black Hawk Down) ... He hit me. It was terrible."
Actor ‘dragged me by my hair’
The brunette daughter of a well-to-do Los Angeles physician, Fleiss said she did not call police because she was on federal parole from her jail term handed down for money laundering and tax evasion.
"Tom (got) very mad at me because I said that I didn't like the movie (Black Hawk Down)...He hit me."
"Because I'm ‘evil Heidi, convicted criminal’," she explained. "The worst thing for me would be to go to prison."
Fleiss testified that in May 2002 Sizemore hit her with his hands in her bedroom, and "dragged me across the room by my hair".
She claimed she had a "black eye and fat lip" the next day.
Star faces separate abuse case
In his opening statement, Sizemore's attorney Michael Fitzgerald told the jury that Fleiss' testimony would be disproved in evidence given at the trial that began Tuesday.
"Tom Sizemore is innocent. Heidi Fleiss is a liar," he said.
Sizemore, whose film credits also include Natural Born Killers, is charged separately with five misdemeanour counts stemming from an alleged incident of domestic violence involving another woman.
Fleiss left prison in 1998 after serving 20 months of a 37-month term for conspiracy, money laundering, tax evasion and attempted pandering (obtain clients for a prostitute) following her arrest after an undercover sting operation in 1995.