Securing the Company Jewels

checklist, a gear, and security icons, illustration

Dear KV,

I am dealing with someone you would call an idiot (a word I cannot use in my work setting) from our IT security department. This “person” has singularly decided to secure our corporate GitHub with many repos and many years of history. Securing something as important as the company’s code is a task I would applaud, if only the person assigned to it had ever used GitHub, or written and deployed software, but, amazingly, the person doing this has done none of those things. Like many of the employees (I hesitate to use the word engineers) in our IT department, this person seems to have arrived with a sort of generic checklist to work from. Whenever our development team asks a question about something this person wants to secure the system, they return a blank look, like a deer in the headlights, or perhaps someone watching an oncoming train. I keep thinking this cannot be how modern security is done, but maybe I am missing something.

Oh Dear, Oh Deer

Dear Oh Dear,

Several years ago, I received a letter from someone who was dealing with a problematic CSO (chief security officer), one who only bought new toys thinking that so long as they shelled out, they would be secure, which led me to coin the term, checkbook security (see https://bit.ly/3PcIAN6). What you have here is a different animal entirely: this is runbook security.

Before I expand on how much I hate this type of thing, let me remind everyone I am a huge fan of documentation, and many of the responses I have written over the years explain why one should write and appreciate documentation.

Just as there is good code and bad code, and good documentation and bad documentation, there are good runbooks and bad runbooks. A good runbook is written in clear prose, describes each step or item clearly, and the best ones give context on why that step matters. In fact, a good runbook can be an excellent educational tool for everyone who uses it. Unfortunately, such a runbook—actually, such documentation in general—is a rare find.

There is also the challenge of securing something as fungible and amorphous as a development workflow. Every piece of software has its own history, warts, and gotchas that make the way it is managed in a source-code system somewhat unique. Even members of the Rust community, who are well known for providing good processes and practices to those developing in that language, cannot predict how a large and complex system, even one in pure Rust, will be built. You can see in the design of their tools and workflows they wish it could be so, and KV often wishes—well, not really wishes, more like screams in anguish—for a system that makes building large systems easier.

We are saddled with all sorts of legacy systems and hodgepodge build workflows, which interact with source-code control in ways that often defy logic and reason.

For now, though, we are saddled with all sorts of legacy systems and hodgepodge build workflows, which interact with source-code control in ways that often defy logic and reason. I am sure we as technologists will keep trying to address these problems; I just hope I live long enough to see one I actually do not hate.

Often the problem with a runbook is not the runbook itself, it is the runner of the runbook that matters. A runbook, or a checklist, is supposed to be an aid to memory and not a replacement for careful and independent thought. But our industry being what it is—a place in which people like to cut corners and dumb things down so that “anyone can do it!”—we now see people take these things to their illogical extremes, and I think this is the problem you are running into with your local runbook runner.

Supposedly, they are versed in security, which is why they were hired, or maybe they just had some ridiculous certification on their LinkedIn page and that is all they needed to get the job—well, that and a pulse. If you are dealing with a Security Runbook Zombie, I probably cannot advocate doing to this person what they do to zombies in the movies, but I can tell you that you have a long road ahead of you.

If I am feeling kind, I would suggest having someone who really understands how your company or development group uses GitHub sit down with the person from your IT security group and walk through the way things work, demonstrate the whole workflow, and then discuss each item in the runbook or checklist or whatever, one by one, to see if it makes sense for securing your system or not.

The key question all good people ask during security interactions remains, “What problem are you trying to solve?” If your zombie cannot answer that, then you have a real problem, and you will probably have to run up the management chain to get a new zombie assigned to work with your team. If you find you can work with your zombie, that is probably better, because eventually, with enough care and feeding—and more brains, of course—you will have someone who understands both security and how it applies to securing what are probably the company jewels.