It was not immediately clear if the imam was the intended victim, although Sunni religious leaders have increasingly become targets in Iraq.
Sahir Jalal, 37, who was at the mosque, said the imam had just begun delivering the sermon when a tall man stood up.
"Then he took out a small rifle from under his jacket and started to shoot," he said.
While violence in Iraq has dropped, the area in and around Mosul is considered one of the last strongholds of the Sunni-backed forces opposed to the US-led military presence in the country and the scene of some recent deadly attacks.
Tal Afar, a mostly Turkoman city, is located along one of the major smuggling routes from Syria to Mosul and has gone through erratic cycles of stability and instability for years.
While Iraqi forces work to assume control of security from US forces, the US military has said that fighters would play on sectarian distrust before national elections are held in January 2010.