The win continues the host's perfect record in the Super Eights stage with their final match against India to be played on Sunday.
New Zealand, having beaten both India and England, but will be watching India's final two games with interest as they could be overtaken on run rate.
Proteas quick Morne Morkel took four for 17 to lead a South African fightback after Lou Vincent (32) and Brendon McCullum (38) had put New Zealand in a commanding position with an opening stand of 68.
Morkel was denied the first five-wicket haul in Twenty20 internationals when umpire Billy Doctrove incorrectly ruled that the delivery that bowled Mark Gillespie in Morkel's last over was a no-ball.
After their strong opening stand New Zealand fell away and they depended heavily on Craig McMillan's unbeaten 48 to post a respectable total.
McMillan and Jacob Oram (15) shared 41 for the fifth wicket, New Zealand's only other stand to note.
In South Africa's reply, Kemp and Mark Boucher (23) put on 65 for the fourth wicket after their side had slipped to 45 for three.
Shaun Pollock (16 not out) helped Kemp complete the job with an unbroken stand of 48 and South Africa reached their target with five balls to spare.