Tutorial: Learning Curve Analysis using DataShop

Held as part of the Third Conference on Learning Analytics and Knowledge (LAK 2013)
in Leuven, Belgium April 8-12


Start Time End Time Subject
9:00am 9:20am Introduction / Overview
9:20am 10:30am DataShop Basics
10:30am 11:00am Break
11:00am 12:00pm Exercise
12:00pm 12:30pm Exercise Review / Wrap up

Learning curve analysis in DataShop


John Stamper, Carnegie Mellon University
Ken Koedinger, Carnegie Mellon University
Ryan Baker, Teachers College at Columbia University
Zachary Pardos, MIT


Learning curve analysis is one way to show learning within educational systems. A learning curve visualizes changes in student performance over time. In this tutorial we will teach the attendees how to perform learning curve analysis on log data. We will explain in detail how to create different types of learning curves, what the curves can show, and how they can be used to improve instruction. Attendees will be given the opportunity to work hands on with actual data to fit student models to data in order to create accurate models for prediction. These models will be based on the Additive Factor Model (AFM), which uses a set of customized Item-Response models to predict how a student will perform for skills in the instruction over opportunities to learn these skills.

This tutorial is enabled by DataShop, which is the world's largest open data repository of transactional educational data collected from online learning courses, intelligent tutors, educational games, and simulations. The data is fine-grained, with student actions recorded roughly every 10 seconds, and it is longitudinal, spanning semester or yearlong courses. As of October 2012, almost 400 datasets are stored including over 90 million student actions which equates to over 200,000 student hours of data. Most student actions are "coded" meaning they are not only graded as correct or incorrect, but are categorized in terms of the hypothesized competencies or knowledge components needed to perform that action. DataShop allows researchers to import data in order to use the provided analysis tools, and to export data from the repository to perform additional analysis. Researchers have analyzed these data to better understand student cognitive and affective states and the results have been used to redesign instruction and demonstrably improve student learning.


Contact John Stamper—john AT stamper DOT org

Equipment needed

Participants should bring laptop computers if available.