Carl McHugh's winner two minutes into injury time at the end of the game gave the Bantams their first victory in League 1 since November - ending a winless run of 13 games in the process.
Despite the last-grasp nature of City's winner though, there was no denying Parkinson's men were worthy winners after they had dominated large parts of the fixture.
Man-of-the-match Adam Reach hit the post in a second period controlled by City before McHugh sent the Coral Windows Stadium into raptures with his late, late intervention.
Parkinson said: "The big feeling for me after this game is relief.
"When we scored the goal, I couldn't take my eyes off the ref as I waited for him to blow the (final) whistle.
"When he blew it, it was a big relief for myself and everybody else connected to the club within the ground.
"Importantly though, I felt we really deserved it tonight (Tuesday).
"I'd spoken before about wanting the players to put in a consistent performance throughout the whole of a game.
"From the first whistle to the last, we were great.
"It was a game where we possibly could have scored more goals in but it was just pleasing to see us get our rewards in the end.
"This victory is a massive boost for everyone at the club.
"Everyone has stuck together - the players, staff and the board - and we have all been working hard to try and get ourselves back on winning form.
"We just knew that if you keep working hard and keep doing the right things, that win will come. That is what the lads have done again tonight and this time they have got their just rewards."
Although the results hadn't been coming for City in recent games before this Tuesday's meeting with Vale, there was certainly no lack of confidence in the Bantams' play against Micky Adams' side.
Dominant throughout, City bossed proceedings for large parts, limiting their visitors to a small number of attacks during the match.
Despite the Bantams' pressure on Vale's goal though, City looked destined to have to settle for a share of the spoils until McHugh popped up with time running out to net his first of the season.
The goal was actually McHugh's first for the Bantams since grabbing City's third in the club's memorable Capital One Cup Semi-Final First Leg victory over Aston Villa around thirteen months ago - an effort very similar to his Vale winner.
Parkinson said: "I'm so pleased for Carlo.
"As I've said before, he has such a big heart.
"He is such a character and a very popular member of the dressing room. He shows tremendous spirit and endeavour every time he plays.
"He showed us that plenty of times last season, none more so than during that Aston Villa game.
"Looking at tonight though, I have to say his goal was probably more important (than his Villa goal).
"There was a real intent in our play. We were camped in their half at times in the second period.
"When (Adam) Reachy hit the post, it does make you start to wonder.
"We kept putting the ball into the box with quality - Gaz's (Gary Jones) set-pieces were right on the money - and when you have players like (Rory) McArdle, (Andrew) Davies, (James) Hanson, (Aaron) Mclean and McHugh attacking the ball, a defence can only cope with so much.
"In the end, we got through."