Why You Need a Software Delivery Machine
Many teams have a clear vision of how they want their software delivery to work. For example, what checks and staging deployments should occur on commits; what approval steps are required before promotion to production; what code needs to be included in new projects and what provisioning should occur on project creation; and what policies matter around license files and security scanning.
But it’s hard to realize that vision. Typical challenges include:
- Growing proliferation of services, meaning many delivery pipelines that can’t easily be changed as one, and many repositories with dependencies, configuration and usage practices getting dangerously out of date.
- Bringing on new developers, due to lack of effective knowledge sharing and lack of automation
- Creating new projects without copy/paste, leading to wasted effort and inconsistency
- Lack of visibility into the whole elephant. What is deployed where? What is at what version? What is happening across the organization? Who should be informed in the case of a production alert, and to what code does it relate?
Atomist aims to solve these problems and deliver a comprehensive solution for delivery. One that you can start with quickly to address a specific need or adopt in full, and that can help smooth the process from commit to production deployment.
The solution to these software problems is more software. These problems can best be addressed together, through greater automation, backed by a model spanning development and delivery. It’s what we call a software delivery machine.
The solution is a framework on which you can develop your development experience, using a common domain model — the Atomist API for software.
When we overcome these problems, we can achieve comparable gains in our delivery process to those we’ve achieved in developer-facing technologies over the past decade in moving from the steam engine of early J2EE to the productivity of Rails, Node and Spring Boot. We can reap the full benefit of re-platforming with modern technologies like Docker and Kubernetes.
An integrated approach gives us the potential to raise our collective productivity so we’re not talking about the dubious benefit of the 10x developer, but the real and durable benefit of the 10x team.
What is a Software Delivery Machine?
A software delivery machine automates the key things your team cares about, whether steps from commit through to deployment or any frequently repeated actions.
Right now you probably have a lot of software delivery implements that do not amount to a machine. CI/CD is necessary but not sufficient. A mishmash of scripts and third party integrations makes an incomplete and brittle solution that wastes valuable team time and leaves dangerous inconsistency.
Today’s teams need something much more ambitious, integrated and powerful.
Think of a software delivery machine as among the most important applications your organization runs. Even if it’s presently not a single application so much as a pile of scripts, build pipelines and duct tape, you already have something in this role. Its pieces are disconnected and built on different technologies. No one is aware of exactly what it’s made of. And hence no one is aware of exactly what’s going on across your organization’s development and delivery, either.
Making the concept of a software delivery machine visible and focusing on improving it is an important step forward, whether using Atomist or home-grown technology.
A software delivery machine can help increase visibility and automation in your team. It can help move hidden actions on individual developer machines into the open where they can be observed by team mates. It can take manual, ad hoc actions, make them repeatable, versionable and visible — ultimately, not merely automated but automatic, triggered by events within the domain of your team.
Your Very Own Software Delivery Machine
Everyone’s ideal software delivery machine is different, because organizations and teams have different requirements and existing assets. But our work with Atomist users has shown common needs and capabilities.
We’re pulling together everything we’ve talked about at Atomist into one compelling solution you can adopt in full or in part or use as a springboard.
Based on our own experience and conversations:
- There’s nothing like the power of programming and real programming languages. Software is eating the world, and software developers are eating software. You want to develop your development experience in a real programming language. Not YAML. Or
- Automations must be testable. You shouldn’t have to wait to see if things break in the real world when changing your development or delivery process.
- Automations shouldn’t be isolated, unrelated point solutions, but should build on a common model.
sdm open source project. It can run anywhere, but is registered with and invoked by the Atomist automation hub.
Your most important policies usually concern how code gets from commit to production. For example, you may want a flow like the following:
In the Atomist software delivery machine, we model this by setting up rules to handle different pushes in different ways. For example: Pushes to
master go through a staging environment to production; pushes to other branches are deployed independently for testing. But if no code or configuration changed, do nothing. Rules are evaluated in order and choose goals to be executed:
Let’s see what happens if the project README is changed. Atomist dispatches an event to the automation. The first rule fires, determining that nothing should be done as this is a Java repo and the change is unimportant:
Rules can access any information about the push, including the branch, the contents of the last commit and the developer who made it. The crucial code invoked by this particular rule is in the MaterialChangeToJavaRepo predicate function, which works with Atomist’s
Project API to check that no Java, JSON, XML or HTML files were changed. Such checks can easily be customized. Ability to avoid build/deploys on immaterial changes can save a lot of developer time and electricity — as can the corresponding ability to draw attention to particularly important changes.
If we make a change to a Java file, we do want a rebuild and redeploy. In this case the second rule fires, setting goals aiming for deployment to test and production:
We can customize many other behaviors in the delivery process, like reacting to a new or closed issue, through registering listeners that react to domain specific events. The following code adds listeners to issue creation and closing and to the creation of a new repo with code in it:
thankYouYouRock function reacts to an issue being closed and uses the Atomist domain model, including access to the chat identity of the person who closed this issue, making it possible to direct message them:
When a developer closes an issue, this will cause them to receive a DM like this in Slack:
We can also benefit from Atomist’s unique ability to comprehend and manipulate the content of repositories to add “autofixes”: automatic changes on a push. The following autofix adds a
LICENSE file if one doesn’t exist, using Atomist’s
This will produce a commit like the following:
This capability enables us to enforce invariants such as our projects always contain a valid LICENSE file or our projects are always linted, as well as to perform more sophisticated changes to configuration and code.
Atomist also enables you to make code changes across many projects: for example to upgrade the version of a library across many projects.
Atomist’s github-sdm software delivery machine, with its domain specific listener types and fluent builder style, is a framework, built on the solid underpinnings of GraphQL and the Atomist API server. Think of it as Spring Boot or Rails for Atomist. It’s not the only way to use Atomist, but it’s a highly productive one.
Hopefully this whirlwind tour whets your appetite for a fuller explanation of what a software delivery machine might enable your team to do. This video provides a deeper dive:
Modern frameworks and deployment technologies have transformed development and deployment for the better. But delivery has become more, rather than less, complex. At Atomist, we’re changing this.
As developers, we want the API for things we work with, so that we can apply our creativity to them and extend them in ways their creators may not have envisaged. Atomist provides the API for software, spanning both your code and delivery process and opening up functionality you work with every day.
What would you automate about how you work if it were this easy? What automations would you hope to use from Atomist and third parties?
Next, we’ll be blogging about aspects of the Atomist software delivery machine, including:
- How to integrate existing technologies such as CI tools and static analysis tools
- Incremental adoption of one use case at a time, like static analysis or autofixes
The github-sdm open source project is on GitHub. And please join the conversation at the Atomist community Slack. I hope to see you there!