Sign In

Communications of the ACM

Contributed articles

A Large-Scale Comparative Study of Beta Testers and Regular Users

A Large-Scale Comparative Study of Beta Testers and Regular Users, illustrative photo

Credit: G-Stock Studio

Beta testing is an important phase of product development through which a sample of target users (potential adopters) try a product ahead of its official release. Such testing is practically ubiquitous; in every kind of company, from medicine to software development, participants test and troubleshoot products to help improve their performance and avoid defects.

Back to Top

Key Insights


Companies should care who their beta testers are. In order to generalize beta-testing outcomes, the population of testers must be as representative of the target users as possible. If not, results of the testing could be biased and fail to capture important product flaws; that is, beta testing for different purposes demands different sets of beta testers. Such aspects of beta testing are often underestimated by software developers; their companies often use any beta tester available, without proper selection, and later without analyzing to what extent the testers were comparable to the population of (targeted) users. Though it was once easier for companies to know their beta testers well,6 this is not the case today due to the vast reach of the Internet and quickening pace of releasing updates and new versions. They should indeed pay more attention to their testers, selecting wisely, because appropriate beta testing is more efficient and economical than later potential failure of a new product.


No entries found

Log in to Read the Full Article

Sign In

Sign in using your ACM Web Account username and password to access premium content if you are an ACM member, Communications subscriber or Digital Library subscriber.

Need Access?

Please select one of the options below for access to premium content and features.

Create a Web Account

If you are already an ACM member, Communications subscriber, or Digital Library subscriber, please set up a web account to access premium content on this site.

Join the ACM

Become a member to take full advantage of ACM's outstanding computing information resources, networking opportunities, and other benefits.

Subscribe to Communications of the ACM Magazine

Get full access to 50+ years of CACM content and receive the print version of the magazine monthly.

Purchase the Article

Non-members can purchase this article or a copy of the magazine in which it appears.
Sign In for Full Access
» Forgot Password? » Create an ACM Web Account