Bringing Industry Back to Conferences, and Paying for Results

https://bit.ly/3lLz4mI July 17, 2020

Computer Science has had a long, friendly, synergistic relationship between industry and academia. For example, when I started going to the International Symposium on Computer Architecture (ISCA, https://iscaconf.org/isca2020/) in the 1970s, 40% of the papers were on real products from industry.

Industry papers fell from 40% in the 1970s to 10% recently at ISCA (https://www.sigarch.org/publication-trends-at-isca/), and even that 10% includes papers based more on industrial research than on industrial products. If these trends continue, the historic bond between computer architecture research and practice could fade, making it harder to understand the problems facing industry and for our research to have impact. When I complained at ISCA 2019 about the lack of papers on real industrial products, ACM SIGARCH chair Sarita Adve assigned me to help fix the problem for 2020.

I thought the only hope was a separate submission process with a separate program committee (PC) whose members believed retrospective papers on industrial products were valuable complements to academic research papers. The PC members also needed to understand company concerns about patent issues or trade secrets may mean some details are not revealed. We tried the following guidelines for industry papers, which differ from regular ISCA and other conferences that have industry tracks:

1. Recruit submissions. Few product architects have permission—let alone time or motivation—to write papers. Given the paucity of product papers, we reached out to nine companies with our top 10 reasons to publish at ISCA (https://bit.ly/3lNuPqY).
2. No intern/sabbatical papers. Papers by visiting students or faculty already appear at ISCA. We required the first, and virtually all, authors must work in industry.
3. Real product papers. We selected papers based on real products, not prototypes developed in industry research labs.
4. Identify products and authors. ISCA uses double-blind reviewing, but industry paper reviewers want to know the actual name of the company and the product and whether the authors are its architects.
5. Later submission deadline. Management must approve industry papers before submission (often involving multiple rounds of redaction), and there can be restrictions about filing patents before submitting a paper. We delayed the paper deadline two months to improve the odds of receiving papers. Besides, reviewing ≈20 papers takes less time than ≈400.
6. Small PC. We limited the PC to nine people (https://bit.ly/3bvOBlV) to facilitate full discussions of the papers at the PC meeting.

The competition was stiff for the 19 papers we received (three recruited), with the majority of papers being strong. They were so strong that Sophia Shao and I are guest-editing a special issue of IEEE Micro (https://bit.ly/32TqV77) for the papers we could not accept. PC members also on the regular ISCA PC found the industry papers very engaging and brought interesting perspectives with different values than the typical ISCA submission. We accepted 5/19 or 26%, only one was recruited (the regular ISCA PC accepted 77/428, or 18%). Each paper proudly bore the label “Industrial Product.”

Figure. The industry track session makes the ISCA program stronger/more exciting.

I was delighted the papers appeared in the honored first session of ISCA. The big question was the audience reaction. The graph shows the main survey question about the industry track (132 responses). I am not sure what other questions would get 80% agreement from skeptical computer architects—free alcohol and ice cream at the ISCA reception? Given the many changes we tried, I was thrilled with these results.

I suspect few of the selected papers would have survived the traditional ISCA process, as reviewers would expect different content and evaluations, so I believe a separate PC is critical to the success of an industry session. I think restricting papers to be on real products by actual industry authors is what made attendees say the ISCA program was stronger and more exciting than in the recent past. The small PC led to more thorough discussions than a typical PC meeting, and the delayed deadline increased our submissions. We intend to follow all six guidelines in the future.

Some conferences already have a version of an industry track (https://bit.ly/2Gt5qCD). Given the

• importance of a good ties between industry and research in computer science,
• decline in participation by industry in many conferences, and
• strong positive reaction from both authors and the audience to this experiment,

perhaps more conferences should institute an industry track?

Back to Top

Comments

Thanks David, for your article on this important topic. I agree with all you say (a standard I don’t achieve with even my own writings). From my experience, having been an ACM member since 1973, the connection between academe and industry has monotonically declined to a very sad state. I applaud efforts to reverse the trend; yours being an excellent role model that I hope will inspire others.

—Robert Akscyn

Back to Top

Yegor Bugayenko: The Remote Revolution Has to be Driven by Output, Not Salaries

https://bit.ly/2XmoqbL May 29, 2020

Across the world, millions of people are giving remote work a go. The COVID-19 pandemic has thrown us all for a loop and forced countless companies to shutter offices, warehouses, and everything else.

Before the pandemic, remote work was already gaining steam in many industries and circles. For workers, it is promised as “liberation” or “freedom” or whatever buzzword makes people feel better.

Some companies see remote as a panacea, too. If everyone’s working remotely, there’s less chance of office politics, right? Costs go down as well because you don’t have to rent, heat, and furnish office space. Blah, blah, blah.

Many companies will realize how difficult remote work is. Monitoring employees remotely is hard. Keeping teams motivated and on task is even more difficult. Maintaining productivity is next to impossible, especially if you don’t adapt your management and compensation models for the reality of remote life.

Working remotely works best when you pay people for results, not by the hour. Many companies are still paying their employees salaries, and they will find out just how hard it is to motivate an off-site employee when they’re paid the same no matter their output.

Think about it; if you will get paid the same whether you code for eight hours or watch Netflix all day, which are you going to do? Sure, a lot of employees will try to be productive; they’ll wake up, work for an hour or two, then take a break.

A 20-minute break can quickly turn into two hours. Some employees will remain productive, and holding people accountable can keep people on task, but as long as the rewards remain the same, productivity will slowly slip.

It is harder to control employees remotely. You can’t walk around the office to make sure people are on task and not goofing off. Nor can you swing by the desk to check in and ask for an update.

Need to schedule a team meeting? It’s easier to gather the team when they’re all in an office rather than scattered across town. Software problems, hardware issues, scheduling conflicts, and all that become more frequent.

So what’s the answer? If people are being paid to deliver concrete, measurable results, a lot of these problems will disappear, or at least be less of a hassle. Don’t pay by the hour, pay by the output.

For one, you don’t have to monitor people if they’re only being paid for production. If a worker sat around watching Netflix all day and didn’t get his work done, that’s not just your problem; it’s also his problem come payday.

People have more incentive to make certain meeting software is installed correctly and their computer is set up properly before a meeting starts. When you pay people by the hour, they get paid even while they installing updates and turning on their mic. When they’re compensated for results, they’re wasting not only your time, but also their own.

Some predict the COVID-19 pandemic will encourage more companies to shift away from traditional office spaces. Working from home or the coffee shop will become the norm because people will realize how wonderful it is.

What’s more likely is that companies will realize how much of a hassle remote work is, and how out-of-tune it is with hourly pay and fixed salaries. Employees may find themselves more worried (rather than less) about their bosses looking over their shoulders or bugging them for updates.

Many companies that had given the work-from-home revolution a go before the pandemic have already scaled back their efforts. Google, Best Buy, and Yahoo are just a few that have found out how tough managing operations remotely is. Many more will learn painful lessons in the months ahead.

If you want to go ahead with the remote revolution, then switch to results-based payment models. Pay people and pay them well when they deliver credible results and outputs. This will provide employees with the motivation to succeed on their own.