How Software Bugs led to ‘One of the Greatest Miscarriages of Justice’ in British History

In a Kafkaesque nightmare come true, nearly 1,000 individuals who ran local post offices in the U.K. were wrongly convicted of stealing money from those operations between 1999 and 2015 as a Fujitsu software system known as Horizon erroneously showed imbalances in their accounts.

The convictions resulted in prison for some of the managers and financial ruin for many held responsible for the missing funds. Those that were not prosecuted were typically fired, resulting in wrecked lives, including four suicides.

Public awareness of what is commonly called both the Post Office scandal and the Horizon Post Office scandal has long percolated throughout Britain but came into sharp focus in early January 2024 when television network ITV aired a prize-winning drama entitled “Mr. Bates vs. The Post Office.” The series portrayed the distress, hardships, and abject disbelief experienced by sub-postmasters (British parlance for local post office managers, also known as sub-postmistresses) as the central Post Office bosses over the years refused to acknowledge any faults with Horizon, and insisted local managers pay up.

The ITV series took its name from Alan Bates, a dismissed sub-postmaster from Wales who painstakingly led 555 sub-postmasters to a 2019 civil case victory against the Post Office in London’s High Court. The court awarded £58 million to the sub-postmasters, much of which went to legal fees.

The case might not have been the monetary win that the sub-postmasters had wanted, but it was a huge moral victory. It served as an indictment not only of the Post Office and of Fujitsu, but of Horizon itself. In his judgment, Justice Peter Fraser noted that “bugs, errors, or defects” undermined Horizon’s reliability and caused discrepancies or shortfalls at branches “on numerous occasions.” The version that the Post Office used from 2000 through 2010—known as “Legacy Horizon”—“was not remotely robust,” he observed.

“Legacy” processed information locally and uploaded it; the later “Online” version, still in use, uploads information for central processing.

With public outrage swelling after the TV drama, and with former U.K. prime minister Rishi Sunak last March describing the convictions as “one of the greatest miscarriages of justice in our nation’s history,” the government in May 2024 dismissed all convictions in England, Wales, and Northern Ireland; it did the same for Scotland in June. It also established a scheme for compensating former sub-postmasters, which launched at the end of July.

To date, no criminal charges have been filed against the Post Office or against Horizon’s supplier, Fujitsu. However, in June 2021 the U.K. government launched a “statutory public inquiry,” in which witnesses can be compelled to testify. The inquiry is ongoing. Like the High Court case, it has been damning of the Horizon software, which is still in use, full of patches.

Bugged and Overburdened

How did the software fail so grievously?

The answer, much of it a matter of public record, dates back three decades. It is rooted in poor coding and testing, worsened by fixes that created new problems, and intensified by a massive expansion of duties.

Horizon is a point-of-sale accounting software system that carries out money-in and money-out transactions at post office branches and creates a record of each monetary transaction on Post Office central computers. It was developed in the 1990s by British company ICL, which Fujitsu acquired. Called Pathway in its early days, it was originally supposed to serve two U.K. government entities: the Post Office, and the Department for Works & Pensions (DWP).

Before Horizon went live, the DWP withdrew. With the government having invested significantly in the project, the Post Office carried on. In what IT expert Jason Coyne (a key witness in the High Court case) described as “scope creep,” the Post Office continued to demand more from Horizon than originally planned, as the organization expanded well beyond the sale of postage stamps and sundries. It added services such as banking withdrawals, lottery ticket sales, driver license and motor vehicle registration and license processing, foreign exchange transactions, mobile phone top-ups, and utility bill payments.

While Fujitsu’s Horizon by and large did its job, sometimes it failed. It was those failures—exacerbated by scope creep but rooted in the project’s beginning—that caused the ruinous financial discrepancies.

The failures included the “bugs, errors, or defects” that Justice Fraser noted in his High Court judgment. He based his findings on evidence presented by individual IT experts from both sides: Coyne, who at the time ran his own Preston, U.K.-based company, IT Group, for the sub-postmasters, and Robert Worden for the Post Office. Coyne pointed out 29 “bugs, errors, or defects” that in his estimation had “lasting financial impact.” As the civil case carried on, the Post Office eventually accepted 21 of them, Fraser ruled.

Coyne, who today calls his IT evidence firm Evolution, discussed his work on the case at length with Communications. He said among others, the bugs in Horizon included:

Double entries
A messaging software bug called the “Callendar Square/Falkirk Bug” (first seen at a post office in the Callendar Square shopping center in Falkirk, Scotland) caused transactions to mistakenly be entered twice. If a customer withdrew £250 from a bank account via a local post office, the information about the transaction transmitted to Post Office central might indicate two £250 withdrawals. The central Post Office would then hold the local sub-postmaster responsible for the “missing” £250. This bug had its roots in faulty messaging software called Riposte provided by a company called Escher Group, Justice Fraser concluded. Riposte itself was buggy. It was a Horizon bolt-on intended to simplify the process of messaging the host computer. In some cases, it failed to synchronize those updates in a timely manner.
No cancellations
While more of the “lasting financial impact” bugs occurred on the Legacy system before the 2010 switch to Online, the latter also had serious flaws that, when they kicked in, would make an innocent sub-postmaster appear to have his or her fingers in the till. The Dalmellington Bug did just this. Named for the post office branch in Dalmellington, Scotland, where it was first noted, unbeknownst to a sub-postmaster it would keep in play a transaction that the sub-postmaster thought he or she had cancelled. It popped up in instances when a sub-postmaster was transferring money to a remote or mobile branch.
Don’t go back to the previous page
Another bug associated with Horizon Online caused cash values to double (or more), to the detriment of the sub-postmaster. The so-called REMM IN bug would record an amount of cash a branch post office had received from headquarters, delivered in barcoded red money bags. When the pouches arrive, the sub-postmaster scanned their barcodes as part of the process of reporting back that he or she has received, say, £4000. However, if, in a cautious act of double-checking, the postmaster hit the “previous” key to make sure his entries were correct, then the entry would record as many times as the sub-postmaster hit “previous” or the back button. As with the Dalmellington Bug, the sub-postmaster would not be aware of the multiple entries, which would trigger a false debt for the mistakenly inflated amount.

In a similar manner, a “REMM OUT” bug also victimized sub-postmasters by having them unwittingly understate the amount of cash they were sending back to the head office.

Bad Beginnings

Even before “scope creep,” the system was destined for trouble from the start.

As most any software engineer would attest, coding errors and bugs happen; it’s a fact of computer life. Yet the degree to which they occurred from the onset of Horizon’s development in the 1990s has astonished more than one expert observer of the case. The theme of “bad coding” coupled with “bad testing” runs through both the High Court case and the ongoing public inquiry.

David McDonnell, who was a member of the ICL Pathway development team in the 1990s, slammed the coding procedures during those years when he testified to the Public Inquiry. “It’s beyond anything I’ve ever seen even in the 25-30 years since that project,” McDonnell said. “Some of the stuff that we found buried in the code was unbelievable…You could see looking at the code, the way it was written, different modules, no standards were being followed. It was a mess.”

McDonnell cited a lack of peer review and criticized the “reverse documentation” of writing specifications after, rather than before, code was developed, to give the appearance of following prescribed rules. “It looks good on paper, but that isn’t the design waterfall flow that should have been followed,” he testified.

McDonnell also described “code decay,” in which code rewritten to fix bugs would adversely affect other parts of the system.

Coyne echoed McDonnell’s observations.

“In the very early days, pre-2000 before it went live, yes, I think there was incredibly bad coding, and there was coding that didn’t appear to be any particular design of specification,” Coyne said.

Coyne dismissed the possibility that, as some observers have suggested, accidental or illicit tampering of sub-postmasters’ accounts via remote access by Fujitsu software engineers created financial imbalances. As Coyne pointed out, remote access is generally a good thing to have in large systems, for support services. Suggestions that Fujitsu or the Post Office created problems this way are a “sideshow” to the real issue of software bugs, he noted.

Both McDonnell and Coyne have alluded to the possibility of many more bugs that have not been confirmed or discovered.

Coyne also noted there is another as-yet-unexplored potential source of bugs: the possibility of flaws within the systems of the big institutions partnered with the Post Office. Those banks, utilities, and other large corporate entities might have some responsibility for some of the imbalances in customer accounts. If that proves to be the case—and there’s no saying it will—then get ready for Horizon, Part II.