Although City looked comfortable for long periods of the second period, an 88th minute winner from Oldham substitute striker Jordan Bove proved enough to send Parkinson's men to a harsh defeat.
Parkinson said: "We never like losing - we always say that.
"This competition isn't really about the money. To make money in this competition you really have to get to the final, which would have been a tall order for us given the size of our squad.
"We do like winning though, and we'll have to take this defeat on the chin.
"There are areas of the pitch we can get better at but there are also so many good things to take into the Yeovil game at the weekend.
"Their goal has come about really from their own real effort on target of the half, so it is a bit of a cruel way to go out, but I don't think we could have done much more in terms of the way we played.
"It is just a shame that the goal came when it did as it gave us no real time to respond.
"The attitude of the players was good and some of the football we played was excellent.
"We had a good period at the start of the second half where we were on top and we perhaps should have got a goal.
"I think if we had scored then we might have gone on to win the game comfortably.
"We have been punished for one lapse. Over the course of the 90 minutes, we have deserved to go through but unfortunately we haven't."
One of the positives Parkinson could certainly take from his side's Oldham JPT tie was the performance of French defender Christopher Routis, who made his Bantams debut at centre half.
Parkinson said: "It was a good debut for Chris.
"You have to bear in mind that the last game he has played for us was against Bradford (Park Avenue) in pre-season, which is quite a long time ago now.
"He has worked hard in training to maintain his fitness levels. We are confident he will be a good player for us because he is a great professional, he's composed on the ball, he's brave, and I am really pleased with him.
"As the season progresses, he could be a big player for us."
Routis was one of five players brought into the team by Parkinson from the eleven that began Saturday's win at Rochdale.
Other changes included starting berths for the likes of Matthew Dolan, Oliver McBurnie and Mark Yeates.
The alterations meant bench roles for James Hanson, Mason Bennett and Billy Clarke, while Billy Knott was an unused substitute. Gary Liddle missed out altogether due to injury.
Parkinson added: "Oli (McBurnie) has done as well as we could have expected on the night.
"He had a good chance in the first half with the header which he has gone in bravely to contest.
"He hasn't really played any football at all this season. Even in pre-season, he didn't get any real time on the pitch because of a few niggles injuries he's had.
"It was a big ask for him tonight (Tuesday) but he will come on a lot for that performance.
"First team football is tough and he was up against a very experienced central defender in George Elokobi tonight but he did a lot of good things.
"We will have to see how Gary (Liddle) is Thursday when we next come back in to see if he has any chance for the weekend.
"With Billy Knott, I wanted to rest him and I believe that was the right call. I also chose not to start with Billy Clarke because I felt (Mark) Yeatesy deserved a start.
"I thought he was one of our better players tonight. He was one of our best players against Coventry City at home on the first game of the season and he has been unlucky not to start many more after that.
"We've had to make some tough selection decisions.
"He has to keep going though. He has to keep producing performances like he did tonight, keep training well, and when the opportunity comes again, whether it be this weekend or next, be ready to take it like he did tonight."