Version 1.6 of the Software Delivery Machine framework is out now! This release is focused on ease of use and scalability.

New features

Some of the noteworthy features added in the 1.6 release are described in the following sections. This release also includes various bug fixes and improvements detailed in the Changelog section later in this post.

Data-defined goal sets

While we at Atomist believe that your delivery flow should be defined in software, we recognize that sometimes the easiest way to get started is with data. To that end, we now support defining the relationships between goals in a data structure as part of your SDM configuration. Using the configure function provided by @atomist/sdm-core, you can return your goal structure. For example:

import { AutoCodeInspection, Autofix, Cancel } from "@atomist/sdm";
import { configure, Version } from "@atomist/sdm-core";
import { Build } from "@atomist/sdm-pack-build";

export const configuration = configure(async sdm => {
    const autofix = new Autofix();
    const build = new Build();
    const cancel = new Cancel({ goals: [autofix, build] });
    return {
        check: {
            goals: [
                [autofix, cancel],
                [new AutoCodeInspection(), new Version()],
            ],
        },
        build: {
            dependsOn: ["check"],
            goals: [build],
        },
    };
});

Each property in the returned object represents a named goal set made up of one or more goals. The top-level property names, check and build in the above example, are used as the name of the goal sets. As shown above, the goals property of each goal set can be either an array or an array of arrays. In the latter case, elements of the inner array are executed in parallel. In either case, the elements of the outer array are executed serially, with later elements dependent on the successful completion of all previous elements. The dependency graph between goal sets is specified using the goal set dependsOn property. In the above example, the build goal set depends on the successful completion of all goals in the check goal set.

Annotated data-defined goal set (courtesy of Jessitron)

Container goals

Recently, CI/CD tools have begun to use Docker containers natively to define tasks to perform. With the proliferation of tools runnable as Docker images, using containers makes setting up many tasks as simple as a quick Google search and copy & paste. While spawning Docker container or Kubernetes jobs has always been possible within an SDM, with the 1.6 release we now provide a container goal that eases the task considerably. Combined with data-defined goal sets, the container goal approach makes for a powerful yet easily comprehensible means for defining your software delivery.

import { hasFile } from "@atomist/sdm";
import { CompressingGoalCache, configure, container } from "@atomist/sdm-core";

export const configuration = configure(async sdm => {
    sdm.configuration.sdm.cache = {
        enabled: true,
        path: "/tmp/atomist-cache",
        store: new CompressingGoalCache(),
    };
    return {
        node: {
            test: hasFile("package.json"),
            goals: [
                container("node", {
                    containers: [{
                        args: ["sh", "-c", "npm install && npm test"],
                        env: [{ name: "NODE_ENV", value: "development" }],
                        image: "node:12.4.0",
                        name: "npm",
                    }],
                    output: [{
                        classifier: "node-modules",
                        pattern: { directory: "node_modules" },
                    }],
                }),
            ],
        },
        docker: {
            test: hasFile("Dockerfile"),
            dependsOn: ["node"],
            goals: [
                container("docker", {
                    containers: [{
                        args: [
                            "--context=dir://atm/home",
                            "--destination=atomist/samples:0.1.0",
                            "--dockerfile=Dockerfile",
                            "--no-push",
                            "--single-snapshot",
                        ],
                        image: "gcr.io/kaniko-project/executor:v0.10.0",
                        name: "kaniko",
                    }],
                    input: ["node-modules"],
                }),
            ],
        },
    };
});

The above example uses the node:12.4.0 Docker image to build Node.js packages and caches the generated node_modules directory for the subsequent Docker build using kaniko. A future blog post will go into more detail on the combination of data-defined goal sets, caching, and container goals. In the meantime, you can check out some examples in the Atomist samples repository.

Caching

As mentioned in the previous section, the SDM framework now supports caching output from prior goals for use in subsequent goals. In addition to the container-goal-specific interface to caching, you can use the cachePut and cacheRestore helper functions to add listeners to any goal to store and retrieve files, respectively. You can find an example of a goal set using caching in the Atomist samples repository.

Job support

We have introduced job support into the SDM to improve the scalability of SDM operations when working on large numbers of repositories or executing operations that are long-running. Now, a command or goal can schedule jobs and tasks using the Atomist API for software, which will manage the scheduling, monitoring, and, if necessary, restarting the jobs and tasks.

Goal creation and configuration

Creating complex goal sets with goals having different fulfillments for different technology stacks has been simplified by the introduction of the createGoals method made available on the SDM instance used in the configure function. The createGoals method takes two arguments: a goal creator and an array of goal configurers. It first calls the goal creator, a function you provide that returns all of the goals your SDM will use. It then calls each of the goal configurers in turn, passing them the goals. Each goal configurer typically adds goal fulfillments and project listeners for a specific technology stack. You can see examples of a goal creator and a goal configurer in the demo-sdm repository.

Changelog

Here is the combined changelog for the following released projects:

The format is based on Keep a Changelog and the projects adhere to Semantic Versioning.

Added

  • Move code transforms to job support. #756 @atomist/sdm@1.6.0
  • Add support for scheduling commands as Jobs. #172 @atomist/sdm-core@1.6.0
  • Add invokeCommand method. 6581460 @atomist/sdm-core@1.6.0
  • Implement container-based job goal. #162 @atomist/sdm-core@1.6.0
  • Introduce createGoals on SDM. #183 @atomist/sdm-core@1.6.0
  • Add x-request-id to apollo requests. caef74a @atomist/automation-client@1.6.1
  • Tell people not to run git-hook manually. #96 @atomist/cli@1.6.0
  • Add statsd metric to indicate WS backoff. d0701de @atomist/automation-client@1.6.0

Changed

  • Always send closed flag and add timeout. #184 @atomist/sdm-core@1.6.0
  • Update Atomist and TypeScript deps. 78ea594 @atomist/sdm-local@1.0.9 5f93892 @atomist/cli@1.6.0

Deprecated

  • Deprecate Fingerprint goal. 4e894f9 @atomist/sdm@1.6.0

Fixed

  • ProgressLog is not populated for code transforms. #745 @atomist/sdm@1.6.0 #163 @atomist/sdm-core@1.6.0
  • Use major version for durable registrations. 46a9200 @atomist/automation-client@1.6.1
  • Fix install behavior. 866638a @atomist/cli@1.6.0

Update path

Please run the following commands to update your SDM dependencies:

$ npm install --save \
    @atomist/sdm@latest \
    @atomist/sdm-core@latest \
    @atomist/automation-client@latest 
$ npm install --save-dev @atomist/sdm-local@latest

Run the following command to update your CLI installation:

$ npm install -g @atomist/cli@latest

Alternatively on macOS, the Atomist CLI can be installed by Homebrew:

$ brew install atomist-cli