We’re witnessing a shift in the way HCI research is done. With the advent of the internet and social media, it is now possible to deploy systems to thousands or millions of potential users. In contrast with the old model of building a prototype in a lab and testing it out on a handful of users, this new style of research — which I call Venture Research — has the potential to reach orders of magnitude more users and achieve significantly more impact.
Yet Venture Research also brings with it new challenges. Foremost is designing new metrics for success. When you can’t bring people into the lab to use your prototype, how do you measure its effectiveness? Many Venture Research projects focus on collaboration or social network-based effects. When a tool’s worth can only be observed “in the wild”, rather than in a controlled lab environment, how do you study it?
One measure of success is adoption. Twitter estimates over 25 million users; Facebook claims over 250 million active users. What made these sites so successful? Are there scientific principles that govern when a social networking site succeeds, and when one fails? Obviously one way to learn is to experiment; Venture Research projects are all about experimenting with social software in the public arena.
However, deploying software on the internet is very different from deploying a prototype in a lab setting. As a result, Venture Research projects call for a different set of skill sets than traditional projects. Our own CoScripter project team has learned all these skills the hard way. For example, the internet is always on; users expect software to be available 24/7. We’ve had to learn how to design our software for continuous availability and stability through new releases and system upgrades. We’ve had to learn how to do technical support, to answer users’ questions (to grow our user base) and capture bug reports from the field. We’ve learned (the hard way) how important good design and usability testing are to a product for which no one reads the manual, but expects to be able to use immediately.
Venture Research takes on a variety of forms. CoScripter and ManyEyes follow a standalone model, deploying independent web applications that hope to grow a large user base on their own. Others leverage existing sites, such as Michael Bernstein’s Collabio on Facebook, or Ed Chi’s work on Wikipedia. Greg Little’s TurKit leverages Mechanical Turk to solve computational problems. What all these projects have in common is that they are tightly coupled to the network. Success or failure of the research is linked to success or failure on the network, with real users.
Join the Discussion (0)
Become a Member or Sign In to Post a Comment