Restoring Industry Participation in Computer Science Conferences

Unlike some fields, Computer Science has had a long, friendly, and synergistic relationship between industry and academia. For example, when I started going to the International Symposium on Computer Architecture (ISCA) in the 1970s, 40% of the papers were on real products from industry.

Alas, industry papers fell from 40% in the 1970s to 10% recently at ISCA, and even that 10% includes papers based more on industrial research rather than on industrial products. If these trends continue, the historically strong bond between computer architecture research and practice could fade, making it harder to understand the problems facing industry and for our research to have impact. When I complained at ISCA 2019 about the lack of papers on real industrial products, ACM SIGARCH chair Sarita Adve overheard and assigned me to help fix the problem for 2020.

I thought the only hope was a separate submission process with a separate program committee (PC) whose members believed that retrospective papers on industrial products were valuable complements to academic research papers. The PC members also must understand that company concerns about patent issues or trade secrets may mean some details are not revealed. We tried the following guidelines for industry papers, which has many differences from regular ISCA and other conferences that have industry tracks:

1. Recruit submissions. Few product architects have permission—let alone the time or motivation—to write papers. Given the paucity of product papers, we reached out to nine companies with our top 10 reasons to publish at ISCA.
2. No intern/sabbatical papers. Papers by visiting students or faculty already appear at ISCA. We required that the first and virtually all authors must work in industry.
3. Real product papers. We selected papers based on real products, not prototypes developed in industry research labs.
4. Identify products and authors. ISCA uses double-blind reviewing, but industry paper reviewers want to know the actual name of the company and the product and whether the authors are it’s architects.
5. Later submission deadline.  Management must approve industry papers before submission (often involving multiple rounds of redaction), and there can be restrictions about filing patents before submitting a paper. We delayed the paper deadline two months to improve the odds of receiving papers.  Besides, reviewing ≈20 papers takes less time than ≈400.
6. Small PC. We limited the PC to nine people to facilitate full discussions of the papers at the PC meeting.

The competition was stiff for the 19 papers we received (3 recruited), with the majority of papers being strong. They were so strong that Sophia Shao and I are guest editing a special issue of IEEE Micro for the papers that we couldn’t accept. PC members who were also on the regular ISCA PC found that the industry papers were very engaging and brought interesting perspectives with different values than the typical ISCA submission. We accepted 5/19 or 26%, only one was recruited (the regular IS CA PC accepted 77/428 or 18%). Each paper proudly bore the label “Industrial Product.”

I was delighted the papers appeared in the honored first session of ISCA. The big question was the ISCA audience reaction. The graph shows the main survey question about the industry track (132 responses).  I’m not sure what other questions would get 80% agreement from the traditionally skeptical computer architects—free alcohol and ice cream at the ISCA reception? Given the many changes we tried, I was thrilled with these results.

In retrospect, I suspect that few of the selected papers would have survived the traditional ISCA process, as reviewers would expect different content and evaluations, so I believe a separate PC is critical to the success of an industry session. I think that restricting papers to be on real products by actual industry authors is what made attendees say the ISCA program was stronger and more exciting than ISCAs of the recent past. The small PC led to much more thorough discussions than a typical PC meeting and the delayed deadline increased our submissions. We intend to follow all six guidelines above in the future.

Some conferences already have a version of an industry track. Given the

• importance of a good ties between industry and research in computer science;
• decline in participation by industry in many conferences; and
•  strong positive reaction from both authors and the audience to this experiment;

perhaps more conferences should consider these guidelines and institute an industry track?

 

David Patterson is a Google distinguished engineer; a professor for the Univesity of California, Berkeley; RISC-V International Vice Chair, and the RISC-V International Open Source (RIOS) Laboratory Director. His newest book is The RISC-V Reader: An Open Architecture Atlas and his best known is Computer Architecture: A Quantitative Approach. He and his co-author John Hennessy shared the 2017 ACM A.M. Turing Award.