The Best of Tools Are Those That Reduce Friction
Note: This is a guest post by Justin Florentine, Director of Software Engineering at Elsevier. Justin shares his perspective on how and why friction happens in software. He presents the perks of using automation and how automating tasks is the simplest way to get the job done right and one of the best ways to incentivizes good habits.
Have you ever talked to a drafter about their favorite mechanical pencil? Spoke to an electrician about their favorite pair of Lineman's pliers? Or a carpenter about their favorite scribing knife? Hooooo boy, I hope you've got a minute!
Craftspeople and tradespeople can and will talk for hours about their favorite tool—the best tool. What they love about it. What it helps them create. How great it is and why it is the absolute best.
But "best," of course, is subjective. When I say best, I mean best for me. The tool that I love to use and am very personally invested in — think stockpile. Professionals who spend a lot of time solving problems in systems tend to fall in love with the tools that help them get "the other stuff" out of their way, remove the friction that slows down progress toward the goal, and stop them from solving problems.
Additionally, they encourage the worker to work the right way. If you've ever tried to do something the right way, but the system you're in provided some kind of resistance, you've experienced this kind of friction. You could've done it the right way, but it wouldn't have been the easy way. If you reach for that tool, and it's not easy to get a hold of because it's under a pile of other things, then maybe it's easier to use the wrong tool. How many of us have knocked something loose using the heaviest thing at hand instead of going to get a hammer? Yep, me too.
The Right Tools For The Right Job
Mechanical pencils produce lines of a consistent size and don't need sharpening. A good pair of pliers cuts, strips, and crimps while an electrician installs fixtures. A sharp and sleek knife leaves crisp lines that can be sawn or planed to. Those tools contribute to efficient problem solving by reducing, eliminating, or automating maintenance work. They reduce the friction between the creator and their goal.
Friction reduction is implemented not just by the tools a worker chooses but also by their space and how they organize their thoughts.
A good craftsperson is continuously revising and improving their workspace, so the system’s many pieces interoperate in ways that sweep away the busywork and allow them to focus on the really tough problems they want to solve. The best carpenters spend much more time designing furniture than they do building it.
Once you have those tools and methods dialed in, you can devote more energy to solving the actual human-facing problem. When you are building a desk, for instance, the desktop is a small problem; the drawers and elements the human interacts with are the big problems you put your soul into.
So, software. Friction happens in software all the time. Sometimes it can be a good thing, keeping us from the siren song of over-engineering. Most devs get over that, though, and friction is a major hindrance from that point on. We've all run into it, and it's not just taking a shortcut. I once worked at a place where a very skilled tech lead overruled a very pragmatic architect who wanted to factor out some functionality from a bloated service into a new, smaller service. There was nothing wrong with any of the ideas, the new service boundaries they were drawing, the APIs being moved, or even the implementation. Everything was neatly packaged by feature, and it could've been done in a day. They didn't do it. They specifically chose not to do it because there was so much friction, both organizationally and operationally, to deploying a new service. What if they had tools that removed that friction?
Keeping The Pace At Elsevier
Now I work at Elsevier, where we sell educational software for training nurses. Our software is web based, integrates with Learning Management Systems, and is deployed in a cloud-native architecture. Specifically, microservices. We have lots of services and applications, and never enough engineers to work on them. We are regularly adding new features, refining old ones, and upgrading infrastructure. Application developers will always push our platform's capabilities and discover new capabilities we didn't even realize we needed.
But hey, we're busy and understaffed, just like you. We don't always have the budget to do the maintenance we need to do, so we have to find ways to make it quicker and cheaper. I'll give you an example: CI/CD. We've had a CI/CD system for the last four years, which was just sorta hacked in there. It's fine. It's a big pile of bash scripts that delegates out to Gradle and Node.
It has worked for a long time, and it's a perfect example of YAGNI (You Aren't Gonna Need It). Except… now we want to shut down the system we're running it on, and boy, it'd be nice to have a proper CI/CD product that can keep up with the times. Something that engineers are more familiar with and less proprietary than a bash script.
We started a project for the new thing, and we realized since the old system did so little, that, ouch— we have a ton of drift and divergence amongst our repos. Some folks have been deleting old branches after merges; some haven't. Some have been squashing merges; some haven't. Some have been upgrading Gradle/Bower/Yarn, etc.; some haven't. You get the picture.
Folks are eager to migrate to the new hotness, and we'd like to use this opportunity to instill a little rigor in our repository upkeep. Clean up your branches. Upgrade Gradle. Double-check your CODEOWNERS file. Make sure your repo settings are sane. When was the last time you scanned for secrets in your git history? These types of "spring cleaning" are types of maintenance work that busy developers really don't want to take the time for. They know they really should, but that's one of the dangers of software—your neglect isn't staring back at you like a dusty workshop. Another engineer isn't going to cut themselves on the knife you didn't sharpen. They're just going to do a git blame and then make fun of you on Slack. But they'll suffer through it, or suffer around it, or even just pass the buck on to the next programmer.
A Virtuous Cycle
If we want to get something done and incentivize good habits, we need to automate as much as possible. We're using Atomist to attach skills to repos before they migrate to the new CI/CD system.
Think of it like a "you must be this tall to ride this ride" sign at a carnival. We're setting down a baseline of repo cleanup you must meet before you enter our shiny new CI/CD system. We like to start off with a reminder; just give the team that owns the repo a nudge in the right direction. Extend the benefit of the doubt. 'Hey, maybe they just didn't think to look for the problem?' We can very quickly enable a skill to detect that and warn a user. This is software, and we operate in a mental and intellectual space. There are no "dusty workshops," so we need to add back in the observability that would come with working in a physical space. So maybe we ping them on Slack? Still haven't gotten to it? Ok, let's add some checks to PRs, and hold up the pipeline till the issue is resolved.
The best part about this is the "here, let me just do that for you" — the automation.
We've had a ton of success using secrets management to purge offenses from git as well as git history. We can enforce consistent branch protection policies across all our repos without nagging repo admins to go in and click on repo settings for "consistency." Even behaviors like setting up auto-rebase for PRs are a non-issue, and engineers are usually happy to have that sorted for them instead of bothering to have an opinion on the matter.
My favorite part of this whole solution is that it's a virtuous cycle — there are so few of those! But now we have a real case where Atomist reduces friction, inspires engineers to look for other opportunities to clean up, and everyone gets the benefits. It all manifests in engineers being more confident that their chores are done properly, and their tools are at hand, in the right place, and ready to go. Now they can think more profound thoughts about their business space and really push the limits of their creativity and capabilities as engineers.