After South Africa resumed at 229 for four they advanced to 252 before Neil Wagner captured the key wicket of Dean Elgar for his best Test score of 140 and the last six wickets fell for 56 runs.
A gripping first Test between New Zealand and South Africa hangs in the balance, with the first session on Day 3 likely to be vital to the result in Dunedin.
But Williamson and Jeet Raval sparked the innings with a 102-run stand, a record second-wicket partnership for New Zealand against South Africa, before Raval went for 52. If he bats for an extended period of time, going on with it and getting a century then the Kiwis should take a first innings lead.
Fortunately for the skipper, a desperate lunge spared New Zealand's blushes.
SA go into the Test series after beating New Zealand 3-2 in the recent one-day global series, where Rabada topped their bowling figures.
"When you play two spinners you obviously want to bowl last but I thought the spinners did a good job on day one of the Test match in terms of a holding role".
NZC confirmed Guptill would miss the rest of this season due to injuries to both hamstrings that ruled him out of the first three ODIs against South Africa.
New images show progress on Hyperloop One high-speed transit system
The 500 meter-long DevLoop , which has a diameter of 3.3 meters is located 30 minutes from Las Vegas in the Nevada desert. A SUPER-FAST train that could travel from Dubai to Abu Dhabi in just twelve minutes is inching closer to reality.
But as the New Zealand bowlers exhausted, South Africa added 90 for the loss of one wicket in the middle session and 76 without loss after tea.
Nicholls looked good before he was the victim of a sensational slip catch by Hashim Amla, also off Maharaj.
New Zealand dropped regular bowling spearhead Southee for the first time in five years to make way for Jeetan Patel. They now trail South Africa by 131 runs with seven wickets in hand.
Du Plessis was dismissed for 52 about 15 minutes before the break when he was caught in the deep by Trent Boult off all-rounder Jimmy Neesham, three balls after he had a lbw decision overturned on review.
Tom Latham (10) had another failure to his name when he nicked a Vernon Philander (1-37) delivery behind the stumps in the sixth over.
New Zealand have the edge at the moment but South Africa are a better side and all three outcomes look equally likely with two days of absorbing Test cricket behind us at the Otago University Oval.