Making Bad Decisions

Decision making can be difficult. There could be many possible paths to follow, each with different advantages and potential drawbacks, and not all consequences may be known up front. Not all decisions are popular adding extra stress for the decision maker. Not all decisions have positive outcomes, but that’s life.

Diagnosing bad decisions in software engineering is even tougher because so many technical decisions are context dependent and there are often complex edge cases to consider. A framework or language may be entirely appropriate in one setting, but inappropriate in another. Similarly, a reasonable high-level framework choice might be badly implemented, doomed by downstream lower-level decisions. Budget and delivery expectations play a huge part in any decision process (e.g., X might be the best option, but there isn’t enough money or time for it). There is one component in software engineering, however, that has universal appeal in every setting and that is the whiteboard. Software engineers love whiteboards. The bigger the better! The more the better! Consequently, most people in technical professions can look at this picture and identify that something went horribly wrong…

… how in the world did this happen, and why wasn’t it noticed until it was too late?

Office Context

Explorys was a medical informatics startup in Cleveland, Ohio and one its secondary competencies was moving offices. An early rite of passage was assembling one’s own Ikea desk, and each office was designed primarily around maximizing seating density, the availability of electrical outlets, and coffee. The last pre-acquisition office was originally built as a modest-sized Sears department store in decades past and was a modern art museum before Explorys moved in. That office had a cavernous interior, terrible acoustics, and a lack of conference rooms such that the whiteboards plentifully employed were mostly of the mobile/wheeled variety. Explorys was acquired in 2015 by IBM to become part of the Watson Health division, and then the unit moved to an IBM office in 2016. This new office was in a multi-tenant commercial building in downtown Cleveland and was a top-down IBM-designed facility.

The new IBM office was professional and attractive, but we were more than a little surprised to find every breakout room looking like the picture above—if memory serves, this happened in probably a dozen rooms over multiple floors. Mounting the monitors in the center of the whiteboards wasn’t a one-off mistake, it was on purpose.

Last Touch Blame Syndrome

It would be easy and obvious to point a finger of blame at the whiteboard installers. The installers were culpable in the sense of drilling the holes and inserting the screws in the mounting brackets, but they were contractors and had the least amount of power in this story, other than the power-tools they used to put holes in the whiteboards. The installation paperwork said that monitors went on whiteboards, so that is where the monitors went. After that, the installers went on to their next job.

Stakeholder Alignment

Stakeholder alignment ideally works bi-directionally: the people affected by a decision know who the decision maker is, and vice versa. In this case, I think it was clear that the decision maker didn’t know what software engineers and related analytic teams did with whiteboards. It’s unfortunate, because all it would have taken was a 15-minute walkthrough of our prior office to see how the whiteboards were being used in the workplace—all of them had lots of writing on them and more than a few had sticky notes and magnets holding index cards, and none of them had monitors mounted on them.

The other IBM offices I had visited post-acquisition didn’t have whiteboards that looked like this, at least that I remembered. Was this supposed to be a joke? Was IBM Real Estate mad at our office? Sometimes a bad decision can at least be understood in terms of original intention before things went sideways, but not so with this one. We were all stumped.

Sound Check – Is This Mic On?

While bad decisions can’t entirely be prevented from forming, they should at least be detectable and addressable. What if the installers were trying to raise a concern? Surely they had installed plenty of whiteboards before and sensed something was non-standard. This raises the question of whether there was a feedback loop, which is typically present in good decisions and absent in many bad decisions. There clearly was not one in this case. There are two broad categories of broken feedback loops:

1. Accidental – The people with feedback don’t have an effective mechanism to provide it, or don’t know where to provide feedback even when a mechanism exists
2. Deliberate – The decision maker has made it clear they don’t want feedback

Large organizations are famous for accidentally broken feedback loops, especially with layers of bureaucracy. To get attention for any issue in this situation requires persistence and an application of the phrase “the squeaky wheel gets the grease” —and sometimes it takes a lot of squeaking. When I discretely asked about why the whiteboards had monitors mounted on them, the answer was effectively “don’t ask,” which made it clear this was the latter category of broken feedback loop. The expression “the nail that stands up gets pounded down” comes to mind.

Ego

The situation we found ourselves in was that—according to some—monitors went on whiteboards, and we were to keep our mouths shut. On top of every other failing in this process, ego now stood in the way of any remediation because to admit that there was a whiteboard problem would have resulted in some form of corporate embarrassment. 

But the engineering team I was on had an executive who was brave enough to insist that full-functioning whiteboards were important to the teams and should be provided. For each affected breakout room, a second set of whiteboards were eventually ordered. Problem solved! The new whiteboards were installed opposite the existing whiteboard/monitor combinations. All of the whiteboard/monitor combinations were left in place and became instant conversation starters for visitors during our time in that office.

This story was resolved with a positive outcome, but it took an executive escalation and a showdown over what most would consider generally non-controversial office equipment.

The Challenge

Bad decisions in large organizations can be more expensive and spectacular due to scale, but even a single significant bad decision in a small organization can be hugely, and negatively, impactful. The dynamics of bad decisions though remain the same for all organizations, and the best treatment is early recognition as the longer a bad decision remains in place the harder it is to unwind. Spock would have advised the rest of the crew of the Enterprise to “live long and prosper, but fail fast and small.”

 

Links

ACM blog on “The Art Of Design Regret” and learning from prior mistakes.

ACM blog on IBM Watson Health – “What Happened To Watson Health?

History of Explorys – Office history (see half-way down this post): http://themeildeal.blogspot.com/2021/01/the-actual-history-of-explorys-part-1.html

 

Doug Meil is a software architect in healthcare data management and analytics. He also founded the Cleveland Big Data Meetup in 2010. More of his BLOG@CACM posts can be found at https://www.linkedin.com/pulse/publications-doug-meil