Dealing With Flakey CI Commands With a Retry Loop in Bash
One of the most frustrating things to deal with in Continuous Integration is flakey commands. Whether it’s flakey tests, or intermittent networking issues, when your build fails for issues outside of your control not only does it cause frustration, it reduces the trust in your CI process.
One strategy for dealing with this type of issue is to introduce some retry logic into your commands. This can easily be accomplished with good old bash.
For example, pretend that I have $FLAKEY_COMMAND
and I want to retry it three times before finally failing my build. I could wrap the whole thing up in a bash loop like this.
counter=1
max=3
$FLAKEY_COMMAND
while [[ $? -ne 0 && $counter -lt $max ]]; do
counter=$((counter+1))
$FLAKEY_COMMAND
done
This script will run my command, if the exit code (the output of $?) is non zero (i.e something went wrong) and my counter is less than three, then it will retry the command. You can increase or decrease the number of attempts by adjusting the max
variable.
This is not a foolproof strategy, but is one approach to handle flakey commands in your CI pipeline.
Thank you for reading! Share your thoughts with me on bluesky, mastodon, or via email.
Check out some more stuff to read down below.
Most popular posts this month
- 2024
- Reinstalling Windows at 1am
- SQLite DB Migrations with PRAGMA user_version
- My Custom Miniflux CSS Theme
- How to Disable Wayland in Debian Testing
Recent Favorite Blog Posts
This is a collection of the last 8 posts that I bookmarked.
- Future Fonts from Blog – Brad Frost
- 21st Century C++ from Communications of the ACM
- Submarines DevCon 2025 Keynote Speech from JoshHaines.com
- How I Use AI: Meet My Promptly Hired Model Intern from Armin Ronacher's Thoughts and Writings
- DeepSeek from Maggie Appleton
- Digital Reality Digital Shock from Christopher Butler
- 10 habits to help becoming a Debian Maintainer from Optimized by Otto
- Tiny corners from Manuel Moreale RSS Feed
Articles from blogs I follow around the net
MusicBrainz Picard identifies songs from *.mp3 files and automatically fixes metadata
In my first attempt to switch from streaming to move back to listening to *.mp3 files, one of the issues I encountered was organization: how to standardize the metadata of the songs? The solution I was familiar with at the time — manually editing each son…
via Manual do Usuário April 24, 2025Google's control of the web could be coming to an end
It's been hard to avoid the US government's antitrust case against Meta lately, since CEO Mark Zuckerberg spent three days in front of the cameras in Congress, testifying about his company's alleged anti-competitive tactics. But another equall…
via The Torment Nexus April 24, 2025$5 million in tokens stolen from ZKsync
An attacker compromised an admin account belonging to the ZKsync Ethereum layer-2 project, which is built by Matter Labs. By doing so, they were able to steal approximately $5 million worth of the ZK token, which the project said wer…
via Web3 is Going Just Great April 24, 2025Generated by openring