In a tie rearranged from seven days earlier, Parkinson's men actually took the lead through Billy Knott in the first period but a set-piece goal in either half from City's South Yorkshire visitors proved enough to send the Tykes through.
Parkinson said: "It was a frustrating way for us to lose the match because we'd spoke before the game about Barnsley's strengths at set-pieces.
"They have a good delivery from Conor Hourihane, plus the height and presence of their three centre halves (Lewin Nyatanga, Alfie Mawson, Marc Roberts) and their two attackers (Marley Watkins, Sam Winnall).
"When the opposition have that threat, you can't give any silly and needless fouls away in dangerous positions.
"On the whole, we got through the game without doing that too much but we got punished from the two that we did give away.
"On top of not giving needless free-kicks away, we have to defend the set-pieces better. That is, ultimately, what has got us knocked out of the competition."
After waiting patiently for his opportunity to feature in Parkinson's first team this campaign, Greg Leigh made his full senior debut for City at left back against Barnsley.
Parkinson added: "Some of the lads who came into the side after not playing too much this season - the likes of Nathan Clarke, Greg Leigh - were tremendous.
"Greg showed what he was all about and put in a very positive performance.
"I've told Greg that his job is to put pressure on James Meredith for that left-back slot. He certainly did that tonight - he was excellent.
"He is a forward-thinking player and he always looks to drive into space. His next touch is always a forward one.
"Considering that he hasn't played first team football for us this season before the game, you have to say that was a very good first appearance for us."
Supporters can hear more from Phil Parkinson after Tuesday's tie by clicking on the video link below or subscribing HERE with Bantams Player HD for the FULL post-match interview.