Online Privacy; Replicating Research Results

http://cacm.acm.org/blogs/blog-cacm/108232
May 11, 2011

Our notions of privacy and security are deeply tied to our social and historical notions of person and place. The aphorism "A man’s home is his castle" captures that notion and its roots in English common law. This Castle Doctrine followed settlers to the colonies and was later codified in the Fourth Amendment to the U.S. Constitution.

Your family heirlooms may be secure in your personal castle, but what of the information about you that lives on the Internet? The legacy of physical place and norms around it are far less relevant.

Equally importantly, we often convolve privacy and security without considering their differences. One needs security to protect private information, but one can have security without privacy, as many world events have shown. Security is a topic for another day; let’s talk about the evolving notions of electronic information privacy.

Back to Top

Riding the Light

That picture of you at a family reunion, squinting into the sun, can rarely be de-limited by a physical location. It might be on disk two, machine nine, rack 23 in a North Carolina data center, but it probably will not be for long.

Instead, information flows freely in radio waves among our wireless devices and on photon beams along the fiber-optic cables that connect the burgeoning network of worldwide cloud data centers. It’s cached, distributed, forwarded, copied, mirrored, and indexed.

All of which suggests that we need to rethink our notions of information privacy, moving beyond concepts rooted primarily in person and place, and considering logical privacy. These issues are complex and emotionally charged for they challenge many of our social, cultural, legal, and economic assumptions. I would not presume to offer a definitive answer here. Instead, let me offer three ideas to stimulate our debate about the future of information and electronic personal management in this brave new world.

Let’s return to that family reunion photograph, captured on a smartphone and posted to a social network site. What might I, as a person in the picture, wish to specify and who else might be involved?

First, I might well like to specify a bounded lifetime for the photograph, after which it would be inaccessible to anyone. Of course, the bound might be infinity, allowing it to remain in the electronic ether forever. That is the current default, as more than one person has learned to their chagrin.

Second, I might choose to define the transitivity of access. I could share the photograph with my extended family, but not allow any of them to share it with their friends. Or I might limit access to an overlapping circle of personal or professional friends, preventing viral propagation. This is challenging because our overlapping spheres of social, professional, and familial influence rarely have hard boundaries, as anyone who has configured their social network privacy settings knows all too well.

The usability of specification interfaces for privacy and security deserves far more attention than it has received. All too often, the only options presented are a broad and vague end-user license agreement that one must accept to use a service or a Byzantine set of confusing service configuration options whose effects are less than obvious. Privacy specifications must be made far simpler and more intuitive.

Third, I might wish to define a claims-based access policy. This is not a binary access specification, but rather a statement that this person or this entity can access this photograph for this and only this purpose. Thus, I might grant my cousin the right to look at this photograph, but not to sell, alter, or combine it with other media.

Ownership, privacy, reputation, and decision-making are intertwined in subtle ways. What if I posed for a reunion photograph but one of my crazy cousins was dancing on the table behind me? Who controls that family reunion photograph—me, the drunken dancer in the background, the photographer with the smartphone, all of us? The shifting nature of social relationships further exacerbates these challenges.

Let me end with another aphorism: "Possession is nine-tenths of the law." In a digital world where images, video, and text can proliferate globally in seconds, we need to rethink what "possession" means.

I don’t have all the answers, but I do have lots of questions.

Back to Top

Ed H. Chi "On the Importance of Replication in HCI and Social Computing Research"

http://cacm.acm.org/blogs/blog-cacm/109916
June 22, 2011

I was asked to serve as a panelist at the CHI2011 conference to discuss the issue of replication of research results. As part of this RepliCHI panel, I wrote an essay arguing that replication isn’t just replication of experiments or rebuilding of systems, but instead is used as an important step in building up greater understanding of a domain. Many panelists, including myself, were surprised when many people showed up at the panel (more than 100?), ready to discuss this seemingly dry academic issue. Here is my essay, slightly edited: One mainstream perspective on HCI is that it is a discipline built upon applied psychological science. "Psychological science" here refers to the understanding of mind and behavior, while "applied" here means that it is the application of approaches of methods, findings, models, and theories from the psychology domain. One has to only look at the CHI annual proceedings to see that it is full of borrowed methods from experimental psychology, a particular approach to understanding mind and behavior based on scientific experimental methods. This approach worked well for HCI, since computers can be seen as a kind of stimuli that is not only interesting, but could augment cognition and intelligence.¹

Experimental psychology is based on the idea that if you design the experiment and control the laboratory setting well enough, you will end up with evidence to believe that the results of the experiment will generalize. These ideas about controlled experiments form the basis of the scientific method. As part of the scientific discovery process, we ask researchers to document the methodology and results, so they can be archived and replicated by others.

But my position is that replication is not the only goal. More importantly, if there are limitations to the study, later experiments might expand on the original experiment to examine new contexts and other variables. In these ways, the idea behind the replication and reproducibility of experiments is not just to ensure validity of the results, but it is also an essential part of the scientific dialog. After all, the reason we value research publications so much is not just because they document and archive the results of the research, but also so that others might stand on the shoulders of giants to reproduce and build on top of the results.

Take, for example, the great CHI 97 Browse Off in Atlanta that aimed to put together a number of hierarchical browsers to see which is the "best." At the event, the Hyperbolic Browser² was the clear winner. While the event was not meant to be a controlled experiment, it was widely publicized. Several years later, the experiment was replicated in a lab setting at PARC³ with the top two performing systems during the event—Hyperbolic Browser and Windows Explorer. Not just once, but twice, under different task conditions!

In the first experiment, the results were at odds with the 97 Browse Off. Not only was there no difference between the browsers in terms of performance, it appears that subject variation had more effect on the results than any other variable.

Further analyses showed there was an interesting interaction effect between the amount of information scent available via the interface conditions and performance, with better information scent resulting in lower retrieval task times with Hyperbolic Browser.

In the second experiment, when restricted to retrieval tasks rather than also including comparison tasks, Hyperbolic Browser was faster, and users appeared to learn more of the tree structure than with Explorer.

What’s interesting is the interpretation of the results suggests that squeezing more information on the screen does not improve subjects’ perceptual and search performance. Instead, the experiment shows there is a very complex interaction between visual attention/search with density of information of the display. Under high scent conditions, information seems to "pop out" in the Hyperbolic Browser, helping to achieve higher performance.

The above example shows there are a number of fundamental problems with viewing experimental results as the end result of a line of research inquiry. Instead, they are often the beginning. Further experiments often shed light on the complex interaction between the mind/behaviors of the user and the system. Replication/duplication of results and further research efforts examining other contexts and variables are not just desirable, but are an important part of the whole scientific exercise.

Back to Top

Back to Top