Multiline Bash in CircleCI YAML: How I almost invented the slashtag
Warning this post contains a lot of YAML
We talk a lot about push and pray at Dagger but I spent the better part of the last few days saying every prayer I know trying to get my CircleCI config to work properly with multi-line bash commands inside of a single YAML block.
It was particularly painful because this was for a job that only runs on git tags and I was trying to publish a new release, so every time it didn’t work I had to remove the git tag, make the tweak to the yaml file, and then push the same tag back out.
I had to look it up, but the steps to remove a remote and local tag are:
git tag -d $TAG
git push origin --delete $TAG
Then do some work, commit it, rebase it, tag it, and force push it out again.
If this was for a project that actually mattered, I think it would have been much more painful because I would not have rebased and force pushed and instead I would have had at least 6 new patch versions of my program with the CI config file being the only thing that changes.
What I was trying to do
A few weeks ago I published a mastodon module that sends a toot out every time I release a new tagged version of a dagger module in my daggerverse repo. This takes advantage of CircleCI tag filtering to only execute this job when I have a git tag.
This function calls my dagger module with a bunch of parameters and I wanted to split them out into multiple lines so that its easier to read.
The first version that I tried did work, but it did not allow me to add the #
symbol because bash (or yaml) was treating this as a comment and breaking. Since I was
trying to add the #dagger
tag to my toots this needed to be fixed. I was able
to get it to work by escaping the tag with \#dagger
. This kind of worked, but
the escape symbol was not escaped, so my toots looked a bit weird.
works
command: dagger -m github.com/levlaz/daggerverse/mastodon call toot
--server="https://hachyderm.io"
--client-id="-3UXCeKKYu3U1z2ZANnONS-Artc-WbMuMRKOuSvzuiI"
--client-secret=env:MASTODON_CLIENT_SECRET
--access-token=env:MASTODON_ACCESS_TOKEN
--msg="I just published a new version of a Dagger module, $CIRCLE_TAG, check it out! https://github.com/levlaz/daggerverse"
breaks because # is not escaped
command: dagger -m github.com/levlaz/daggerverse/mastodon call toot
--server="https://hachyderm.io"
--client-id="-3UXCeKKYu3U1z2ZANnONS-Artc-WbMuMRKOuSvzuiI"
--client-secret=env:MASTODON_CLIENT_SECRET
--access-token=env:MASTODON_ACCESS_TOKEN
--msg="I just published a new version of a #dagger module, $CIRCLE_TAG, check it out! https://github.com/levlaz/daggerverse"
works but /# is not escaped
command: dagger -m github.com/levlaz/daggerverse/mastodon call toot
--server="https://hachyderm.io"
--client-id="-3UXCeKKYu3U1z2ZANnONS-Artc-WbMuMRKOuSvzuiI"
--client-secret=env:MASTODON_CLIENT_SECRET
--access-token=env:MASTODON_ACCESS_TOKEN
--msg="I just published a new version of a /#dagger module, $CIRCLE_TAG, check it out! https://github.com/levlaz/daggerverse"
What didn’t work
Then I remembered something about the |
symbol allowing to do more stuff so I
tried that. This didn’t work because my module kept complaining that I was not passing
any parameters.
command: |
dagger -m github.com/levlaz/daggerverse/mastodon call toot
--server="https://hachyderm.io"
--client-id="-3UXCeKKYu3U1z2ZANnONS-Artc-WbMuMRKOuSvzuiI"
--client-secret=env:MASTODON_CLIENT_SECRET
--access-token=env:MASTODON_ACCESS_TOKEN
--msg="I just published a new version of a /#dagger module, $CIRCLE_TAG, check it out! https://github.com/levlaz/daggerverse"
Next, I remembered that maybe I needed to escape them somehow so I added some escape symbols. I could not find any info on how to do this in the docs, but this SO post gave me a hint about using double slashes, so I tried that.
command: |
dagger -m github.com/levlaz/daggerverse/mastodon call toot \\
--server="https://hachyderm.io" \\
--client-id="-3UXCeKKYu3U1z2ZANnONS-Artc-WbMuMRKOuSvzuiI" \\
--client-secret=env:MASTODON_CLIENT_SECRET \\
--access-token=env:MASTODON_ACCESS_TOKEN \\
--msg="I just published a new version of a /#dagger module, $CIRCLE_TAG, check it out! https://github.com/levlaz/daggerverse"
This did not work, it still complained about not passing parameters. Then I tried single slashes. No luck. Then I tried to run the command locally and it worked just fine. I was completely lost and now we were in our 5th iteration of doing the force push rebase tag prayer.
What did work
After staring at my screen for a bit, I decided to try one last thing before
just accepting that all of my toots were going to have \#
in them, perhaps I could
have started a new trend! Slashtag?
I decided to remove all the extra indentation on the subsequent parts of the command.
command: |
dagger -m github.com/levlaz/daggerverse/mastodon call toot \
--server="https://hachyderm.io" \
--client-id="-3UXCeKKYu3U1z2ZANnONS-Artc-WbMuMRKOuSvzuiI" \
--client-secret=env:MASTODON_CLIENT_SECRET \
--access-token=env:MASTODON_ACCESS_TOKEN \
--msg="I just published a new version of a #dagger module, $CIRCLE_TAG, check it out! https://github.com/levlaz/daggerverse/releases/tag/$CIRCLE_TAG"
This worked! And I still have no idea why. Its possible I am just stupid, but I have no idea what the combination of bash, yaml, and the CircleCI engine actually wants from me.
I’m dreaming of a world where no one has to deal with this yaml hell anymore. We’re halfway there, and if you follow me on mastodon you’ll be the first to hear about it, without the slashtag.
Thank you for reading! Share your thoughts with me on bluesky, mastodon, or via email.
Check out some more stuff to read down below.
Most popular posts this month
- 2024
- Reinstalling Windows at 1am
- SQLite DB Migrations with PRAGMA user_version
- My Custom Miniflux CSS Theme
- How to Disable Wayland in Debian Testing
Recent Favorite Blog Posts
This is a collection of the last 8 posts that I bookmarked.
- Give Your Spouse the Gift of a Couple's Email Domain from mtlynch.io
- Have smart glasses finally hit an inflection point? from The Torment Nexus
- The McPhee method from the jsomers.net blog
- Pluralistic: LLMs are slot-machines (16 Aug 2025) from Pluralistic: Daily links from Cory Doctorow
- Pluralistic: Bluesky creates the world's weirdest, hardest-to-understand binding arbitration clause (15 Aug 2025) from Pluralistic: Daily links from Cory Doctorow
- Just a Little More Context Bro, I Promise, and It’ll Fix Everything from Jim Nielsen’s Blog
- The Futzing Fraction from Deciphering Glyph
- Sit On Your Ass Web Development from Jim Nielsen’s Blog
Articles from blogs I follow around the net
Issue 91 – GDP on the blockchain
The regulator set to take on primary crypto oversight is down to a single Commissioner, and new pro-crypto PACs focus on installing more Republicans in the midterms
via Citation Needed August 27, 2025V&A East Storehouse and Operation Mincemeat in London
We were back in London for a few days and yesterday had a day of culture. First up: the brand new V&A East Storehouse museum in the Queen Elizabeth Olympic Park near Stratford, which opened on May 31st this year. This is a delightful new format for a mu…
via Simon Willison's Weblog: Entries August 27, 2025Generated by openring