Is your application security improving, staying the same, or getting worse? How do you prevent changes from introducing new vulnerabilities? While there are many ways to scan for vulnerabilities, the resulting noise can be overwhelming, making it hard to see whether things are getting better or worse. Scanning continuously, addressing new critical vulnerabilities as they arise drives constant improvement in security posture — a "ratcheting up" of security over time.

At Atomist, this is what we set out to address with our latest feature, which prevents only new critical vulnerabilities from getting deployed. Instead of drowning in a sea of reported vulnerabilities, see how each pull request will affect the overall security risk of a project and block changes that add new critical vulnerabilities, moving steadily toward more secure applications.

The principles behind our new approach:

  • 🧑🏽‍💻  Detect vulnerabilities early — identify during development
  • ⚡️  High signal, low noise — raise alerts only when needed, always in the relevant context
  • 📈  Ratcheting up — constantly improve security in each incremental change
  • 👥  Security is a team sport — support collaboration between security and developers
  • 🙌  Celebrate success — recognize when security improves

The benefits:

  • Detect new vulnerabilities in pull requests
  • Alert developers directly in the pull request only when new critical vulnerabilities are found
  • Report how changes affect security across your projects

While teams have different approaches to paying down existing security debt, new vulnerabilities must be rooted out before they ship. The encapsulation of "new" is the pull request, so that's where we focus.

Properly evaluating a pull request requires far more than diffing package manifests. We also need to consider the deployment process. For containerized applications, in order to assess vulnerabilities, we have to look at the image that will be run. That is the only source of truth, as Dockerfiles and other intermediate artifacts in the repository may be altered before being turned into the executable container image.

What should the pull request be compared to in order to determine whether new vulnerabilities are being added? We compare to the target branch that your pull request wants to be merged into. But because we deal in truth (container images), we compare against the image built from the target branch (usually your default branch — ours are called main).

It's helpful to know whether changes make your codebase worse. It's also useful to know if you're about to make production less secure, so we compare pull requests against the last deployed image as well. Together, you get a clear perspective on whether a change is going to make production worse and whether that has already been fixed in main and just needs to be rebased.

Alerting developers directly in the pull request

It's crucial to keep signal high and noise low for developers. A lot of noise can come from low and medium-severity vulnerabilities that don't need urgent developer attention. Only new critical and high severity vulnerabilities introduced in the pull request are flagged in an Atomist GitHub check. You choose whether that fails the build or not — it's configurable.

Instead of a firehose of all of the vulnerabilities found in the image, we show only those that are new, with a summary of the CVE to show which vulnerability crept in and how to address it.

As humans, we work better when we also get positive feedback. When a pull request removes vulnerabilities, that's something to celebrate. Fixed vulnerabilities are shown in the GitHub check including a summary of the specific CVEs fixed.

Reporting how changes affect security

Our app shows the security status of all open pull requests across all repositories. AppSec teams and other security-minded engineers can use this to see which projects and teams might need help addressing newly added vulnerabilities.

What next?

This feature is available now in the Atomist app. Learn more about the feature and get started setting it up.

This new vulnerability feature adds to our Dockerfile Best Practices and Base Image features that help you lint Dockerfiles, ensure you're using supported base images and get repeatable image builds with base image tag pinning.