Learning and Evidence Analytics Framework Bridges Research and Practice for Educational Data Science

Learning analytics (LA) as a research discipline focuses on multiple perspectives of understanding and supporting educational activities utilizing collected log data. To do so at a national and even international level, educational technology platforms that enable gathering users’ interaction traces and digitally generated artifacts must store data in a standardized format. In Japan, the government initiated the GIGA School project in 2020, which installed more than nine million tablet PCs and high-speed Internet access at compulsory education institutions (elemental and middle schools). Such infrastructure enables the collection of educational data and analysis with the aim to improve educational practices in each school. With standardized data logging, it is possible to aggregate data from all schools and to generate educational Big Data that can support evidence-based policy-making and research at a national level.

LEAF is an integrated technology framework that incorporates learning analytics methods and tools and can be linked to any existing learning management systems in different institutions.

At the Learning and Educational Technology Research Unit at Kyoto University, we developed the Learning and Evidence Analytics Framework (LEAF),¹ an integrated technology framework that incorporates LA methods and tools and can be linked to any existing learning management systems in different institutions. The accompanying figure provides an overview of the framework. As illustrated in the figure’s Part A, teachers and learners can conduct daily educational activities and access various AI-driven services utilizing the LEAF system to enrich their learning experiences and outcomes. Teachers can upload any learning materials in PDF format on BookRoll, the e-book platform, and learners can access them through standard Web browsers on their PC or mobile devices (as shown in Part B). The learning interactions are logged using standard xAPI statements and then processed to create various data-driven services in LEAF. It is then presented to the users within BookRoll or in a learning dashboard called Log Palette.

Figure. Learning and Evidence Analytics Framework (LEAF).

The five tenets supporting data-driven teaching and learning practices with the LEAF system includes the following:

1. Prepare the infrastructure for data-driven sustainable educational practices;
2. Support learning for all by using learning log data;
3. Foster self-directed learning skills with learning logs;
4. Enhance interpersonal communication and social learning by using learning log data; and,
5. Share data and evidence.

Our development agenda aims to overcome the challenges in realizing these tenants, which is informed by technology research as well as contributes to investigating the impact in the educational sector.

Educational Explainable AI Tools (EXAIT) is one of the services linked to LEAF.² EXAIT is used to learn English and mathematics at the Japanese junior high and high school levels. For English, the tool recommends appropriate extensive reading materials based on students’ vocabulary exposure to prior reading materials. For mathematics, it aims to detect learners’ “stuck” points based on analyzing digital pen stroke data and students’ self-explanations. In addition to visualizing the learning logs, the evidence extraction function in LEAF can compute the effectiveness of interventions during daily activities as a step to promote evidence-based practices with daily log data.

In Japan, the LEAF system is implemented on the national educational cloud, which is accessed by 20 public schools from seven prefectures.

Another service linked to LEAF is the GOAL system,³ developed to provide approaches to learner modeling and support for self-directed educational activities using learners’ own data synthesized from multiple data sources. LEAF also integrates cutting-edge technologies such as blockchain to facilitate validating and sharing of learning logs across institutions and utilizing logs for algorithmic grouping for cooperative learning scenarios.⁵

In Japan, the LEAF system is implemented on the national educational cloud, which is accessed by 20 public schools from seven prefectures. Additionally, instances are implemented locally in 10 universities reaching more than 20,000 students in the K-16 context for their daily learning activities.² Internationally, LEAF is adopted by teachers in collaborating universities in Taiwan, India, and Hong Kong⁴ and enables a sustainable way to continue collecting educational Big Data in international contexts. It pursues cutting-edge research on educational data science utilizing daily educational activities and aims to support evidence-based learning and teaching practices internationally. The teachers gain access to their own class data and researchers among them also utilize the data to analyze and report findings as academic publications.

The LEAF approach and its implementation at scale from Japan is only possible with collaboration among researchers, practitioners, and policymakers. While the GIGA school provided devices and infrastructure, LEAF provides actual applications to evolve a data-driven educational ecosystem that is unique. The research group is engaged in data collection and standardized logging policymaking. It can be adopted by industries that create online learning tools and services.

A formal council on Evidence-driven Education (EDE) was formed in 2021. EDE provides both a research forum for the different Japanese stakeholders to collaborate with LEAF and a commercial forum to adopt LEAF at the institutional and national levels through a subscription-based financial model. That large-scale adoption of the LEAF system highlights its flexibility and low threshold. It is used by teachers from various disciplines and sociocultural backgrounds. The East Asian and Oceanian community of educators and researchers can explore, adopt and have a local implementation of LEAF in different countries.⁵ Our research aims to create an educational data ecosystem to continuously improve practices by involving different stakeholders who utilize data-driven services in their daily learning context.

Acknowledgment. This research work is partially supported by the following research grants: NEDO Special Innovation Program on AI and Big Data JPNP18013, JPNP20006 and JSPS KAKENHI 20H01722, 22H03902.