SDM 1.4 released

To support the release of Project Uhura, we are happy to announce the release of SDM 1.4. Read more about Uhura in Ryan's introduction post.

In addition to the SDM framework we also released a new project called sdm-pack-analysis which underpins Uhura and provides a very flexible approach to scanning projects for common delivery patterns and scheduling goals and goal fulfillments based on scanned results. More details on sdm-pack-analysis will be part of another blog post.

New Features

Besides many small and big fixes and improvements, the following noteworthy features were added in the 1.4 release:

Preferences

A simple key-value store implementation has been added to the SdmContext to store and retrieve preferences in command handler or goal executors.

See following code snippet on how to use it to store a preference in a command:

sdm.addCommand({
    name: "EnableDeployment",
    intent: "enable deploy",
    listener: async ci => {
        await ci.preferences.put(
            "deployment.enabled",
            true,
            { scope: PreferenceScope.Sdm });
    },
});

The preference can then be used to write a push test to evaluate whether deployGoals should be added to the goal set:

sdm.whenPushSatisfies(
    pushTest(
        "deploy enabled",
        async pi => {
            return pi.preferences.get<boolean>(
                "deployment.enabled",
                { scope: PreferenceScope.Sdm, defaultValue: false});
        })).setGoals(deployGoals);

Pluggable Goal Scheduling

One of strenghts of the SDM framework is its ability to distribute goals onto different runtime environments. Our customers use this to schedule goals as Kubernetes jobs and containers on Amazon ECS and Google Cloud Build, just to name a few..

With this release, we improved the extension points to hook in customized goal scheduling. Take a look at the GoalScheduler interface. An SDM can have zero to many registered GoalScheduler instances.

With this, we also improved the KubernetesGoalScheduler with much better clean up support as well as support for Services.

Services

A very commong feature request we got was to support Services when scheduling goals. A service can be a OS process or Docker container a goal needs to run successfuly, e.g., running mvn test might require a MongoDB instance to be available..

Support for services depends on the way you schedule your goals. The following code example uses our enhanced KubernetesGoalScheduler to add a MongoDB container into the test goal k8s job.

A service can be defined as follows:

export function mongo(tag: string = "latest"): K8sServiceRegistration {

    const container: DeepPartial<k8s.V1Container> = {
        name: "mongo",
        image: `mongo:${tag}`,
        imagePullPolicy: "IfNotPresent",
        ports: [{
            name: "http",
            containerPort: 27017,
            protocol: "TCP",
        } as any],
        resources: {
            limits: {
                cpu: "100m",
                memory: "512Mi",
            },
            requests: {
                cpu: "100m",
                memory: "256Mi",
            },
        },
    };

    return {
        name: "mongo",
        service: async () => {
            return {
                type: K8sServiceRegistrationType.K8sService,
                spec: {
                    container: container as any,
                },
            };
        },
    };
}

Adding the MongoDB service to the test goal can simple be done by calling testGoal.withService(mongo());. Now, your tests can connect to MongoDB at http://localhost:27017.

Dynamic Parameters

SDM commands prior to 1.4 had to be registered with a fixed set of parameters. With 1.4 a command handler can ask for additional parameters during the execution of the listener function.

Here's an extension to the previous command to enable deployment which asks for an additional parameter specifying the target cluster name:

sdm.addCommand({
    name: "EnableDeployment",
    intent: "enable deploy",
    listener: async ci => {

        const cluster = await ci.promptFor<{ name: string }>(
            {
                name: {
                    description: "Target environment to enable",
                    required: true,
                    type: {
                        kind: "single",
                        options: [
                            { description: "Prod", value: "production"},
                            { description: "Test", value: "testing"}],
                    },
                },
            });

        await ci.preferences.put(
            `deployment.enabled.${cluster.name}`,
            true,
            { scope: PreferenceScope.Sdm });
    },
});

Goal Signing

In order to protect the goals as managed by an SDM in your workspace, those goals can now be signed with a private key and verified against a set of trusted public keys. All that is needed for this is a private/public key pair.

This mechanism can also be used to establish trust between different SDMs running in your workspace.

New CLI Commands

The atomist config command has been improved to allow users to log in via the CLI and request an api key.

Additionally we added commands to create new workspaces and to configure Git SCM resource providers to simplify the creation and management webhooks.

Changelog

Here is the combined changelog for the following released projects:

The format is based on Keep a Changelog and the projects adhere to Semantic Versioning.

Added

  • Make full clone an option on CodeInspection Goal. #643 @atomist/sdm@1.3.0
  • Add mergeOptions method. 939c04c @atomist/sdm@1.3.0
  • Stream files during code transform. #585 @atomist/sdm@1.3.0
  • Add PreferenceStore interface. #658 @atomist/sdm@1.3.0
  • Add services to a FulfillableGoal. e5e09df @atomist/sdm@1.3.0
  • Add branchAwareCodeTransform to make it target branch aware. #659 @atomist/sdm@1.3.0
  • Support parameter prompting from command listeners. #661 @atomist/sdm@1.3.0
  • Support @Value in parameter declaration . #648 @atomist/sdm@1.3.0
  • Enhance the typedoc within CodeTransformRegistration. #665 @atomist/sdm@1.3.0
  • Add parameters to EventHandlerRegistration. 230db78 @atomist/sdm@1.3.0
  • Provide a useful implementation of onTransformResults. #666 @atomist/sdm@1.3.0
  • Add conditional command registration. #676 @atomist/sdm@1.3.0
  • Introduce goal signing and verification. #678 @atomist/sdm@1.3.0
  • Add types that help you specify what your Configuration is. #679 @atomist/sdm@1.3.0
  • Adds Gitlab repo targets for commands. #683 @atomist/sdm@1.3.0
  • Add PreferenceStore implementations. #93 @atomist/sdm-core@1.3.0
  • Support parameter prompting from command listeners. #95 @atomist/sdm-core@1.3.0
  • Introduce goal signing and verification. #100 @atomist/sdm-core@1.3.0
  • Use a declared type for the configuration. #101 @atomist/sdm-core@1.3.0
  • Provide a dummy id to InMemoryProject. #438 @atomist/automation-client@1.3.0
  • Allow boolean and number parameters types in command requests. #446 @atomist/automation-client@1.3.0
  • Set WebSocketLifecycle on configuration. 6c009bf @atomist/automation-client@1.3.0
  • Provide a way to globally modify chat messages. #470 @atomist/automation-client@1.3.0
  • Add getContentBuffer method to File. #476 @atomist/automation-client@1.3.0
  • Test failure case in doWithFiles. #471 @atomist/automation-client@1.3.0
  • Allow configurationValue without path. 128f8e4 @atomist/automation-client@1.3.0
  • Add Namespace support for Gitlab. #477 @atomist/automation-client@1.3.0
  • Configure FilePreferenceStore in local mode. #224 @atomist/sdm-local@1.0.5
  • Add uhura to seed list. a160911 @atomist/sdm-local@1.0.5
  • Add install command to search and install an SDM extension pack from an NPM registry. #b706d70 @atomist/cli@1.2.0
  • Add provider and workspace create commands. #61 @atomist/cli@1.2.0
  • Add provider config command. #64 @atomist/cli@1.2.0

Changed

  • Populate the SHA on projects when loading them. #652 @atomist/sdm@1.3.0
  • Make goal scheduling more plugable. #656 @atomist/sdm@1.3.0
  • Make LogSuppressor the default for logInterpreter. 6a921e2 @atomist/sdm@1.3.0
  • Deprecate lower case enum values. f556259 @atomist/sdm@1.3.0
  • Improve startingPoint handling in generators. #663 @atomist/sdm@1.3.0
  • Deprecate paramsMaker. #667 @atomist/sdm@1.3.0
  • Accept extensions starting with . in isMaterialChange. #674 @atomist/sdm@1.3.0
  • Make K8 goal scheduling more extensible. #90 @atomist/sdm-core@1.3.0
  • Move k8s goal launching into extension pack. ccb6fbc @atomist/sdm-core@1.3.0
  • Only one worker for goal jobs. 119fea6 @atomist/sdm-core@1.3.0
  • Remove axios use in postWebhook. #107 @atomist/sdm-core@1.3.0
  • Allow matchIterator to be used to change matches. #442 @atomist/automation-client@1.3.0
  • Various fixes to better support targeting in command handlers. #441 @atomist/automation-client@1.3.0
  • Add support for latest Chooser API contract. 17549fd @atomist/automation-client@1.3.0
  • Update failure logging for failed repo creations. #461 @atomist/automation-client@1.3.0
  • Fall through to no creds for cloneUrl. b31ba59 @atomist/automation-client@1.3.0
  • Remove axios usage and use HttpClientFactory. #464 @atomist/automation-client@1.3.0
  • Update dependencies. d68fef3 @atomist/automation-client@1.3.0
  • Update all deps including graphql-code-generator. #474 @atomist/automation-client@1.3.0
  • Improve extension pack generator. #236 @atomist/sdm-local@1.0.5
  • Deploy k8s-sdm as part of atomist kube. #65 @atomist/cli@1.2.0
  • Add dry run and print current context to kube command. #67 @atomist/cli@1.2.0

Deprecated

  • Deprecated createGoal(). 8b27e81 @atomist/sdm@1.3.0
  • Deprecate lower case enum values. f556259 @atomist/sdm@1.3.0
  • Deprecate paramsMaker. #667 @atomist/sdm@1.3.0
  • ChainTransforms swallows exceptions. #668 @atomist/sdm@1.3.0
  • Add parameters to EventHandlerRegistration. 230db78 @atomist/sdm@1.3.0
  • Add support for latest Chooser API contract. 17549fd @atomist/automation-client@1.3.0
  • Deprecated Chooser type. 60739fc @atomist/automation-client@1.3.0

Removed

  • Remove atomist webhook handling from generators. 498bc1c @atomist/sdm@1.3.0
  • Remove addAtomistWebhook from see generator params. 4ebe315 @atomist/automation-client@1.3.0

Fixed

  • Errors in machine function can get swallowed and ignored. #654 @atomist/sdm@1.3.0
  • Make lazy project loader work for generators. 2e0ed3e @atomist/sdm@1.3.0
  • Do not swallow exceptions thrown by transforms in generators. #664 @atomist/sdm@1.3.0
  • ChainTransforms swallows exceptions. #668 @atomist/sdm@1.3.0
  • Errors in machine function can get swallowed and ignored . #449 @atomist/automation-client@1.3.0
  • Do not swallow exceptions thrown by transforms in generators. #465 @atomist/automation-client@1.3.0
  • Fix typo that crashed it. #475 @atomist/automation-client@1.3.0
  • WebSocket can be destroyed before message sent. #483 @atomist/automation-client@1.3.0
  • Disable TSLint on generated types. #485 @atomist/automation-client@1.3.0
  • Credentials on CodeTransform invocation are not valid. #222 @atomist/sdm-local@1.0.5
  • Local mode does not work if NODE_ENV=production. #228 @atomist/sdm-local@1.0.5

Update Path

For this release, please run the following commands to update your SDM installation:

$ npm install --save-exact \
    @atomist/sdm@1.4.1 \
    @atomist/sdm-core@1.4.1 \
    @atomist/automation-client@1.3.0 
$ npm install --save-exact --save-dev @atomist/sdm-local@1.0.5

Run the following command to update your CLI installation:

$ npm install -g @atomist/cli@1.3.0

Alternatively the Atomist CLI can be installed by Homebrew:

$ brew install atomist-cli