Trailing by an early Lois Maynard goal for Halifax, Parkinson's men fought back with two second half goals within three minutes of each other shortly after the restart to finish off their sprinted Vanarama Conference opponents.
With their hosts fully playing their part in an entertaining encounter, the Sunday lunchtime tie had the feeling of a true traditional F.A Cup contest.
In truth, Halifax forced City to deep dig at times but Parkinson's men eventually showed their greater skill and mettle in both boxes to secure their progress in the competition.
Parkinson said: "Halifax gave us the exact sort of challenge we were expecting from them.
"They are a strong, physical team and they looked to test us on long throw-ins and their other set-pieces.
"They have three central defenders in their team and two others who started their careers in that position, so they are a big and powerful team.
"Unfortunately for us, we got off to the worst possible start. Despite that though, I still felt we had some good passages of play in the first half.
"We got in some good crossing positions, we just lacked that bit of quality in and around the box to get us a goal.
"We said to the lads at half time that if we make it a battle for 45 minutes in the second half, we'll be going out. We needed to be calm in possession and make our quality count.
"It was a big test for us at half time - questions were being asked of us as a group really.
"We had 3,500 fans here and we were one-nil down to a side two leagues below us, but I don't think you could have asked much more of the players given the way they responded in that second period."
City's two goals after the restart came within the first eight minutes of the second period through Jon Stead and Filipe Morais - both of which was created by influential half time substitute Billy Clarke.
Parkinson added: "When you are not in the team, it is always disappointing for any player. The way you do your talking though is out on the pitch, and Billy (Clarke) has certainly done that today (Sunday).
"The two finishes were top class really from Jon and Filipe - great composure shown by the both of them. Steady's was a really calm and composed finish, and I'm delighted for Fil as well.
"He was worked so hard since he's come here and he is getting better and better. Even when he's not in the team, his professionalism is there for all to see.
"To get the winning goal today will be a huge boost for him."
With City looking to see out the tie with time running out, Halifax tried to put the Bantams under enough late pressure to force a reply. City showed their bravery and commitment to successfully defend their advantage, however, leading to Parkinson praising the contributions of skipper Stephen Darby and his deputy.
Parkinson said: "Stephen (Darby) was great today. He was driving forward for us down that right wing supporting Filipe and he also put himself on the line to protect his goal time and time again - it was a captain's performance.
"I thought Davo (Andrew Davies) was back to his best today as well - he was a colossus. Almost every time Halifax put the ball inside our box, it was Andrew who was putting his head on it."