The deputy, Tynychbek Akmatbayev, had been visiting the tuberculosis hospital attached to Prison No 31, about 25km northwest of the capital Bishkek, when the violence erupted on Thursday.
"The deputy Akmatbayev died from gunshot wounds," said a spokesman for the Justice Ministry's prisons service who asked not to be named. He could not immediately say how the prisoners acquired weapons.
Two other members of his entourage were killed and the head of the prisons service, Imatulla Polotov, was taken to hospital with serious injuries. His fate was unclear with conflicting statements about his condition.
President Kurmanbek Bakiyev ordered an investigation into the killings after meeting Prime Minister Felix Kulov and top law enforcement officials in Bishkek.
Shooting accompanied the hostage-taking, a spokesman for the prisons service said, adding that guards had been withdrawn to the perimeter fence.
A riot at another prison earlier this week, prompted by a protest against poor living conditions, forced the staff to evacuate the grounds and security forces to surround it.
President Bakiyev has ordered an
investigation into the incident
The prisons service said security was being tightened at prisons across the country.
Kyrgyzstan, a poor and mountainous Central Asian state bordering China, has been unstable since violent protests ousted former President Askar Akayev in March.
It was not immediately clear if the hostage-takers had made any demands, but Akmatbayev was involved in negotiations on Wednesday at the other prison, also near Bishkek. That prison remained surrounded by security forces on Thursday.
The riot in the first prison, which houses 2000 prisoners, broke out on Tuesday. Kyrgyzstan has around 18,000 prison inmates.