The match was tied at 1-1 after 90 minutes, and the game went straight to penalties, but the young Derby players kept their nerves to win the shoout-out 4-2.
Derby U23s came into the match having won their last league game against Leeds United 7-1, but only three plyers kept their spot in the team; Joseph Bateman, Ethan Wassall and Jordan Brown, with the visitors using the cup-tie as an opportunity for squad rotation.
The visitors dominated the possession from the start, but it was the home side who scored first, with Jordan Hendley‘s ninth minute header looping over Derby goalkeeper Josh Barnes.
The visitors had struggled to break down the Clay Cross defence until the dying minutes of the first-half - Cameron Cresswell‘s shot was saved, but the ball fell into the path of Alex Babos who rounded the keeper from the rebound and slotted home.
Clay Cross gave it their all defensively in the second half and were able to see the game out with the score at 1-1, but the effort they put in was not enough in the end as the Derby won the penalty lottery.
The visitors slotted home all four penalties they took, whilst a Josh Barnes from Grant Mitchell‘s effort, and Tom Torrington‘s strike hitting the crossbar respectively denied Clay Cross their third and fourth.
Ant Lyman, head coach of Clay Cross Town, was delighted with his team‘s efforts, he said: "The lads did tremendously - obviously if we would have come out and attacked them, then they would have picked us off because they are a lot better then us.
"We fully deserved what we got from the game - moving forward, we were never going to win the competition, so it is not that harsh that we have gone out tonight.
"Is it better for us to have lost heroically on penalties than gone somewhere else and got thumped?
"If someone would have said to me before the game 'you are going to lose on penalties tonight', I would have taken it."
Pat Lyons, U23 coach at Derby County, also acknowledged the Clay Cross performance: "They were very good out of possession and made it difficult for us.
"It was a great experience for our lads to come out here and see a different side of the game, as the players usually play on pristine pitches.
"It was good to get over the line in the end, but we made hard work of it and didn‘t create enough opportunities, however it is good to get through to the next round."
Derby, who last won the competition in 2011, will face New Mills FC in the quarter-finals.