They See What You See

Andrew Conway’s and Peter Eckersley’s Viewpoint "When Does Law Enforcement’s Demand to Read Your Data Become a Demand to Read Your Mind?" (Sept. 2017) was an important contribution to the ongoing debate over electronic backdoors, whereby a backdoor is a means for accessing and exfiltrating user information not specifically authorized in advance by users. Here, I would like to outline several key aspects of that debate that also need to be addressed.

Although Conway and Eckersley did discuss the possibility that law enforcement could gain access to our most private thoughts, they did not mention a crucial near-term technology through which this exfiltration could happen. Within the next 10 years, "hologlasses," or holographic glasses, are projected by Apple, Facebook, Google, Microsoft, and Samsung, along with numerous startups, to become almost as common as cellphones are today, as reflected in the scale of their investment in its development. A backdoor in hologlasses could enable a "we see and hear what you see and hear" capability that would provide extraordinary insight into what users are thinking, as well as how they are behaving online and even in the physical world.

Also not mentioned was a legislative proposal that could facilitate mandatory backdoors for Internet of Things devices. In 2016, Senators Richard Burr (R., North Carolina) and Dianne Feinstein (D., California) introduced legislation—The Compliance with Court Orders Act—in the U.S. Senate to mandate providers of information products and services also provide unencrypted information on IoT devices to the government pursuant to court order. The result could be "Nothing is Beyond Our Reach," or no information is beyond the reach of law enforcement, likewise pursuant to court order. Similar legislation has been adopted in Australia, France, Germany, the U.K., and other countries but so far has had only limited effect because these countries are not sufficiently powerful individually to enforce sanctions against large multinational foreign-domiciled IT providers. However, if Burr-Feinstein does indeed become law, then these countries might be more able to pursue mass surveillance domestically, as IT companies could lose much of the legal grounds they would need to resist.

Conway and Eckersley also did not mention a near-term technology that might be used to implement highly secure backdoors in IoT devices by requiring that each device have a different public key that could enable government security services to take over the device.¹ Even if hackers penetrated the security of a government-installed virtual machine for a device, they would gain no lasting advantage hacking additional devices.

Finally and most important, no mention was made of a technology proposal¹ that could ameliorate some of the negative effects of mass surveillance, whereby citizens’ most sensitive information is stored on their own devices, provided personal IoT devices include protection against self-incrimination. By storing sensitive information on these devices, that information could be protected from the kind of efforts Conway and Eckersley identified.

Carl Hewitt, Palo Alto, CA

Back to Top

Authors Respond:

We generally agree with Hewitt who offers definite specific instances of the general issues we covered. Virtual reality, in particular, is, as he implies, a valuable window into the mind that also involves important technical and legislative dynamics. What to do about it is a complex question we did not attempt to answer in our Viewpoint beyond framing its context for a wider societal discussion we consider essential.

Andrew Conway, Melbourne, Australia, and Peter Eckersley, San Francisco, CA

Back to Top

Bring ‘Diseased’ Software Up to Code

We agree with Vinton G. Cerf’s advice in his Cerf’s Up column "Take Two Aspirin and Call Me in the Morning" (Sept. 2017) that we all practice better "cyber-hygiene" but must quarrel with the continued use of public health as a metaphor for cyber security. If we as computing professionals intend to improve the cybersecurity of our critical infrastructures, rather than merely tolerate their current "diseased" state, we must think differently. We thus propose a return to an older metaphor for software, likening its structures to physical structures and its architecture to the architecture of physical buildings. Such thinking suggests we consider how to build software that will not fall over when attacked or build it from weak materials unable to bear expected stress.

Software we rely on for critical functions (such as controlling medical devices, delivering electrical power to households, and guiding automobiles) must conform to an appropriate set of constraints, just as physical structures conform to building codes before they can be occupied. A third party must be able to certify conformance to these constraints, just as building inspectors certify buildings.

These codes are best developed by those who build the systems, not by government, though governments might use them once they are in place. An industry-consensus building code, with third-party assessment of conformance, can help the marketplace reward those who build systems with fewer vulnerabilities.

Over the past few years, with support from the IEEE’s Cybersecurity Initiative and the National Science Foundation, workshops have been held to begin to develop such building codes for medical-device software and for power-system software.^2,3 In addition to these draft codes, related promising developments include Consumers Reports’ collaboration with the Cyber Independent Testing Laboratory (http://cyber-itl.org/) to develop methods for publicly rating software products, and UL’s (http://www.ul.com) development and use of a standard for certifying cybersecurity assurance of products.

Treating software security as a public health problem is not likely to lead past the decades-old ideas of aftermarket vaccines, antivirus, and quarantine. Providing evidence that software is at least free of specified classes of vulnerabilities covered by an appropriate building code can yield a more effective market incentive for companies to produce the cyberinfrastructures we all need—and that are up to code.

Robert K. Cunningham, Lexington, MA, Tom Haigh, Minneapolis, MN, Carl Landwehr, New Buffalo, MI, and Alfonso Valdes, Urbana, IL

Back to Top

Author Responds:

It is always a pleasure to hear from Carl Landwehr with whom I have had a long acquaintance and for whom I have great respect. An interesting challenge with his building/architecture metaphor relates to the way software is often constructed these days by incorporating (vast) libraries of code reflecting, perhaps, uncertain provenance. There is also the uncertainty of software interactions across the network that may never have been tested until a chance encounter leads to a breach. None of this invalidates the building-code metaphor but might make it more difficult to establish that the ensemble meets the desired code standards and properties. I am, in fact, very interested in the development of programming aids that will do a much better job of assessing source code against desirable properties of attack resistance and identifying potential sources of weakness. Until we have reliable ways of producing the kind of code Landwehr et al. and I likely agree we need and want, we may also want to argue that infected or vulnerable devices ought not to be naively tolerated and that owners (and suppliers) bear at least some responsibility for observing diligent software hygiene.

Vinton G. Cerf, Mountain View, CA

Back to Top

A Neuron Net Is Also a Turing Machine

Leo Corry’s article "Turing’s Pre-War Analog Computers: The Fatherhood of the Modern Computer Revisited" (Aug. 2017) described the Turing machine as a purely mathematical notion. While Corry’s argument was persuasive, there is indeed a direct connection from Turing’s construction of the Turing machine to the Electronic Discrete Variable Automatic Computer (EDVAC) in the 1940s via the McCulloch-Pitts model of the brain. For example, during the discussion portion of a 1951 talk by John von Neumann, neuroscientist Warren S. McCulloch described the influence of Turing’s original 1936 paper, saying:

"… I came, from a major interest in philosophy and mathematics into psychology with the problem of how a thing like mathematics could ever arise—what sort of a thing it was . The attempt to construct a theory in a field like [neurophysiology], so that it can be put to any verification, is tough … it was not until I saw Turing’s paper that I began to get going the right way around, and with [logician Walter] Pitts’ help formulated the required logical calculus. What we thought we were doing (and I think we succeeded fairly well) was treating the brain as a Turing machine."⁴

McCulloch’s and Pitts’s 1943 paper emphasized the equivalence of artificial neuron nets to Turing machines, saying:

"It is easily shown: first, that every [neuron] net, … can compute only such numbers as can a Turing machine . This is of interest as affording a psychological justification of the Turing definition of computability and its equivalents, Church’s λ-definability and Kleene’s primitive recursiveness."

Moreover, in the first draft of the design for the EDVAC, von Neumann explicitly tied the computing elements for the EDVAC to McCulloch’s and Pitts’s model of a neuron, saying:

"Following W. Pitts and W.S. MacCulloch [1943] … we ignore the more complicated aspects of neuron functioning … It is easily seen, that these simplified neuron functions can be imitated by telegraph relays or by vacuum tubes … We propose to use them accordingly for the purpose described there: as the constituent elements of the devices, for the duration of the preliminary discussion … The element which we will discuss, to be called an E-element … which receives the excitatory and inhibitory stimuli, and emits its own stimuli along a line attached to it. … In all this we are following the procedure of W. Pitts and W.J. MacCulloch."⁵

Since McCulloch and Pitts had shown that neuron nets are universal computing machines in the sense of the Church-Turing thesis, the same connection to universal computing machines would apply to the EDVAC.

Brad Barber, Arlington, MA

Back to Top

Author Responds:

My entire argument was about Alan Turing’s own views prior to the war, not about his influence on later developments. But a more general point I wanted to make was that scientific and technological ideas develop historically and that what happens later sometimes misleads us when we try to understand what happened earlier on. This may also be the case with McCulloch’s very interesting, retrospective testimony, which by all means deserves a critical eye.

Leo Corry, Tel Aviv, Israel

Back to Top

Back to Top