The War Against Spam; and More

http://cacm.acm.org/blogs/blog-cacm/78121

A decade ago, email spam was a dire problem. Annoyances flooded most inboxes. Any attempt to read your email started with deleting the crud that had leaked through your defenses.

Many predicted the problem would only get worse. A few predicted that email would be dead in just a few years, the filters would be overwhelmed, the war would be lost, and email readers would be buried under an avalanche of spam.

Today, email spam appears to be a solved problem. A 2003 study put response rates at 0.005%. A 2008 study where the authors infiltrated a major spam botnet found response rates had fallen to under 0.00001%, with only 28 sales out of 350 million messages sent. Spam filters appear to have forced down response rates three orders of magnitude in five years. Spammers have fought back with misspellings, adding additional text to emails, trying to customize each email sent, and many other tricks to evade detection, but their increasingly complicated efforts have not been able to outwit the filters.

My own experience is that email spam has become a non-issue. Despite prostituting my email addresses undisguised across the Internet, despite receiving hundreds of spam messages daily, nearly zero make it to my inbox. The ones that I do see typically are borderline spam, from companies and small businesses sending to a small list rather than the mass splattering of true email spam.

Amazingly, the drop in response rates from 2003 to 2008 may be close to making spam an unprofitable enterprise. There is a substantial amount of effort required to attack and manage a botnet of 1 million compromised machines that can cheaply send 12 million messages per day. Huge email campaigns that attempt to work around spam filters require sophistication to devise and run. Email address lists have to be purchased and maintained. It appears to be getting to the point that even massive and complicated spam efforts like the Storm botnet generate surprisingly low revenues for what appears to be the work required.

What is your own experience with email spam? Do you think the spam war been won? Or are these declarations of victory premature?

Back to Top

Reader’s comment:

I would argue that your declaration of victory is premature, but not for the obvious reason that spam in the inbox has been reduced. In that respect, spam in my inbox, and my spam folder as well, has gone way down in recent years, and that is welcome.

However, two problems remain. First and foremost, there is still the problem of false positives. I still have to check my spam folder because the filters will occasionally falsely flag a legitimate email. Once I do find a desired email, I can flag it as non-spam to teach the filter and add the sender to a white list, but that is reactive. If you’re receiving hundreds of spam emails a day, as you wrote, I imagine more than a few false positives slip by you.

Additionally, as an entrepreneur, sending email from a new company like mine that hasn’t established itself with the many filters out there [is] very time consuming and inefficient. If there was a proactive way for a legitimate sender to register itself and either post a bond or pay e-postage, I think that would clean up a lot of email inboxes. I know e-postage proposals haven’t gotten very far in the past, but if the spam response rate is now down to 0.00001% then the postage can be a lot lower as well.

The second reason we can’t declare victory just yet is the very fact that the spammers and their resources, both botnets and humans, remain alive and well. If email spam continues to become less and less profitable, then they will simply send spam in other forms such as on Twitter, Facebook, etc. Individual computers continue to get infected and people still foolishly click on requests from Nigerian princes. Unfortunately, we have to continue to apply technological fixes to our networks and teach people not to be so gullible.

Believe me, I wish we could declare victory, but we’re not there yet.

—Will Hartmann

Back to Top

Michael Bernstein’s "Clay Shirky: Doing work, or Doing Work?"

http://cacm.acm.org/blogs/blog-cacm/72609

MSN usability researchers were stumped. Their usability lab had tested just about every aspect of its MSN portal and had been pleased to find that it consistently scored higher than its competitors. Yet a user base didn’t flock to MSN—the portal simply could not attract and retain as many users as it wanted. Then Clay Shirky relayed the million-dollar question: Were these tasks that users actually wanted to do? Or were these highly usable aspects of the site going to remain unused because nobody wanted to use them? There was a gaping hole between usability and usefulness.

In a keynote delivered to this year’s ACM Conference on Computer Supported Cooperative Work (CSCW), author and academic Clay Shirky captured this question in a distinction between Work and work. Work (Capital W) is what we have considered for years; your boss tells you to do something, you do it, and you get paid. By contrast, work (little w) is motivated by inherent interest and is generally unpaid. Think of the difference between an Encyclopedia Britannica editor doing Work and a Wikipedia editor doing work during spare hours. Big Work drives the economy; little work drives the Internet. Big Work builds skyscrapers; little work generates a half-million fan fiction stories about Harry Potter.

Clay argued that user-testing techniques developed over the past 25 years for Work no longer apply for work. We shouldn’t be asking, "Can you complete the task?" but rather "Are you motivated to do it in the first place?" Excel needs usability testing because people are forced to use it for Work; technology for work instead needs to understand users’ underlying motivations.

"Usability is an important refinement technique when you have a good idea, but it is a horrible determiner of utility on a grander scale."

Extrapolating on my own here: Usability is an important refinement technique when you have a good idea, but it is a horrible determiner of utility on a grander scale. (Sure, pay me $10 for a lab study and I’ll use anything for an hour!) Usability is a local hill-climbing algorithm. We need techniques to make and evaluate that miraculous motivational leap, whether it’s derived from the design process or social science. Develop that and you could save thousands of man-hours developing tools that nobody will ever want to use.

Back to Top

Reader’s comment:

Jared Spool, for years, talked about compelled shopping tasks in which people were actually given money to buy things on Internet sites, and could not. He really wanted to solve the usability problem, but also realized that decoupling the motivational issues with usability is difficult. Little ‘w’ work vs. Big ‘W’ Work suggests that we are going to have to dig much deeper into this issue than we had before.

—Ed H. Chi

Back to Top

Erika S. Poole’s "Death and the Digital World"

http://cacm.acm.org/blogs/blog-cacm/72837

At the Computer Supported Cooperative Work 2010 conference, 13 Ph.D. students received invitations to participate in the Doctoral Colloquium, an event in which new scholars discuss their work with a panel of experts. In addition to being a great opportunity for students, the Doctoral Colloquium highlights some of the most exciting work in the field from promising young scholars. In particular, I couldn’t help but notice that the students invited to this year’s event presented work highlighting the deeply human side of information technology.

For example, imagine you’re a parent who has suffered the unthinkable: Your child has died. How do you cope with such a traumatic, painful, and disorienting experience? For some parents, information technologies can play an important role in the grieving and mourning process. Yet how are bereaved parents using technologies to grieve and mourn? If we were to design technologies that help people cope with grieving and loss in meaningful and respectful ways, what would they look like?

I had the opportunity to speak with Mike Massimi, a Ph.D. candidate at University of Toronto who’s examining these questions for his thesis work. To understand how technology plays a role in modern grieving, Mike is working extensively with two community organizations in the Ontario area that provide social support to parents who have suffered the loss of a child. His next step is to create meaningful, appropriate, and respectful technologies that help bereaved parents mourn and remember their lost children.

Death and dying are experiences as old as humanity, and have been studied by scholars in other disciplines for centuries. Yet technology researchers and designers are just now starting to come to grips with how to design with end-of-life experiences in mind. If you’re interested in learning more about this topic, Mike co-hosted the first workshop (ever!) focused on death and the digital world at the ACM CHI 2010 conference. You can see more info about the workshop at http://www.dgp.toronto.edu/~mikem/hcieol/.

Back to Top

Back to Top