Ian Duncan has written a great post on CI orchestration called No, Really, Bash Is Not Enough: Why Large-Scale CI Needs an Orchestrator. It does a good job of distinguishing between the simple cases where bash and make really are good enough for CI, and when you actually need a full-featured CI system.
I am talking to teams where CI is a load-bearing piece of infrastructure. Teams where 20 or 50 or 200 engineers push code daily. Teams where a broken CI pipeline doesn’t mean one person waits a few extra minutes; it means a queue of pull requests backs up, a deploy window gets missed, and product timelines slip. Teams where CI time is measured in engineering-hours-lost-per-week and has a line item on somebody’s OKRs.
It also leans heavily on one of my favorite papers, “Build systems à la carte” by Mokhov et al. From the discussion of that paper:
The real takeaway is not that bash is bad. It’s that the design space of build systems has structure, and that structure has been studied, and that the properties you care about (minimality, correctness, support for dynamic dependencies, cloud caching, early cutoff) correspond to specific architectural choices that live at a level of abstraction bash cannot express. When you write a build pipeline in bash, you are either implementing one of the twelve cells in the Mokhov-Mitchell-Jones matrix (poorly, by hand, with strings and exit codes), or you are living in the
busycell and rebuilding everything every time.
It’s a long read but a good one, go check it out.