Ali Larijani, the parliament speaker, said: "Out of 273 deputies who received ballots to take part in the vote of confidence, 138 voted for him, 112 rejected his nomination. We have 20 abstentions and three did not take part at all."
Mahsouli had previously been nominated for oil minister, but he withdrew after being criticised for his lack of experience.
Critics argue that Ahmadinejad chose Mahsouli in order to strengthen his position ahead of presidential elections in June next year, a claim Mahsouli denied.
"The issue of me being in Mr Ahmadinejad's [election] campaign group is not true," he said. "And I have never received a single rial in loans either from a public sector or a private sector bank."
The interior ministry is in charge of organising elections.