Expert Finding For Collaborative Virtual Environments

While computer-supported collaborative virtual environments have been successfully applied to revolutionize distance learning, distributed design, and collaborative analysis and planning (see Ragusa and Bochenek’s introduction to this section), a fundamental challenge of these systems is establishing the right teams of individuals during interactive problem solving for consultation, coordination, or collaboration. Motivated by our use of place-based collaborative environments for analysis and decision support [3], we created Expert Finder and XperNet, two software programs that automatically profile topics of interest and expertise associated with employees based on employees’ tool use, publications, project roles, and written communication with others.

Figure 1 illustrates Expert Finder in action. In this case a user types in the keywords “data mining” and Expert Finder replies with a rank-ordered list of employees whose expertise profile, inferred from a variety of evidence sources, best matches this query. Evidence includes the frequency of documents published by an employee on this topic, contents of any published resume, and documents that mention employees in conjunction with a particular topic (for example, corporate newsletters). In the latter case, information extraction technology is used to detect names within unstructured documents. These names are then correlated with topic areas in the documents.

Despite low human inter-subject agreement, empirical evaluations [1] comparing 10 technical resource managers’ performances with Expert Finder on five specialty areas (data mining, chemicals, human-computer interaction, network security, and collaboration) demonstrated that Expert Finder performed at 60% precision and 40% recall when appropriate data was available. By “precision,” we measure the degree to which a staff member found by Expert Finder is considered expert by humans. By “recall,” we mean the degree to which a priori human-designated experts are found by the Expert Finder (that is, the number of actual experts found divided by the total number of experts reported, and the number of actual experts found divided by total number of experts, respectively).

In contrast to the query-based Expert Finder tool, XperNet [2] focuses on finding expert communities of practice using clustering and network analysis techniques. Networks of individuals with related skills and interests are created by processing information about staff project information, publications, and personal Web pages. Expertise indicators (for example, explicit reference or citation, network centrality) as well as counter-indicators (for example, being a member of the administrative staff) are used to assess the level of expertise of a particular individual. The underlying algorithm uses both dynamic cluster merging to create core clusters as well as expansion algorithms that “grow” the network using project and other organizational information to identify additional cluster members.

Figure 2 shows XperNet support for visualizing groups and ranking individuals according to their level of expertise, enabling users to query, retrieve, and navigate expertise networks. As with the Expert Finder evaluation, XperNet performance was benchmarked against human performance. Approximately 70% of the top 10 automatically identified experts were in the manually identified list. Precision dropped about 10% when computed over the top 20. Looking at the top 30, approximately 75% of the experts were identified automatically with approximately 50% accuracy. With increasing availability of online expertise, our research continues, focusing on the ability to automatically characterize, assess, and locate experts.

Back to Top

Back to Top

Back to Top

Back to top