Table of Contents
- What is DataShop?
- Getting Data In
- Accessing DataShop
- Project / Dataset Administration
- Citing DataShop and Datasets
- Filtering Data
- Export: Getting Data Out
- Using Other Tools
- Contact Us
The student-step rollup table aggregates data by student-step: each row represents a student attempting to complete a step. Within each sample, rows are ordered by student, then time of the first correct attempt (“Correct Transaction Time”) or, in the absence of a correct attempt, the time of the final transaction on the step (“Step End Time”).
Knowledge components are not shown by default. To include them, click the checkbox "Knowledge Components" at the left of the screen.
To display the Student-Step Rollup report:
- Click the Learning Curve tab at the top of the screen.
- Click the subtab Student-Step Rollup.
A student-step pair can appear on more than one row. This can happen if the step has more than one knowledge component associated with it (in which case the row is a duplicate except for the knowledge component field) or if the student saw the same problem more than once—there you would see the Problem View number increase.
As on the Export page, you can export your data using the export button. The Student-Step Rollup export includes only your selected sample(s), and reflects the chosen knowledge component models; however, it includes all knowledge components and students within those samples. To include a subset of knowledge components and/or students, define a new sample using the sample selector, and include only that sample.)
|Row||A row counter.|
|Anon Student ID||The student that performed the step.|
|Problem Hierarchy||The location in the curriculum hierarchy where this step occurs.|
|Problem Name||The name of the problem in which the step occurs.|
|Step Name||Formed by concatenating the "selection" and "action". Also see the glossary entry for "step".|
|Step Start Time||The step start time is determined one of three ways:
|First Transaction Time||The time of the first transaction toward the step.|
|Correct Transaction Time||The time of the correct attempt toward the step, if there was one.|
|Step End Time||The time of the last transaction toward the step.|
|Step Duration (sec)|
|Correct Step Duration (sec)|
|Error Step Duration (sec)|
|Incorrects||Total number of incorrect attempts by the student on the step.|
|Hints||Total number of hints requested by the student for the step.|
|Predicted Error Rate (model_name)|
See the Student-Step Rollup Example for a visual description of how step times, step durations, and correct step durations are calculated.
“Predicted Error Rate” and how it's calculated
Predicted error rate is the probability of the student making an error (incorrect action or hint request) on a step, as predicted by the Additive Factor Model algorithm.
- Υij = the response of student i on item j
- θi = coefficient for proficiency of student i
- βk = coefficient for difficulty of knowledge component k
- γk = coefficient for the learning rate of knowledge component k
- Τik = the number of practice opportunities student i has had on the knowledge component k
- Κ = the total number of knowledge components in the Q-matrix
- The Τik parameter estimate (the number of practice opportunities student i has had on the knowledge component k) is constrained to be greater or equal to 0.
- User proficiency parameters (θi) are fit using a Penalized Maximum Likelihood Estimation method (PMLE) to overcome over fitting. User proficiencies are seeded with normal priors and PMLE penalizes the oversized student parameters in the joint estimation of the student and the skill parameters.
The intuition of this model is that the probability of a student getting a step correct is proportional to the amount of required knowledge the student knows, plus the "easiness" of that knowledge component, plus the amount of learning gained for each practice opportunity.
The term "Additive" comes from the fact that a linear combination of knowledge component parameters determines logit(pij) in the equation.
You can view model parameter values and see measures of how well the AFM statistical model fits the data on the Model Values report (a subtab of Learning Curve).
Student-Step Rollup Example
This example demonstrates how DataShop calculates step start time, step end time, step duration, and correct step duration for a student on a series of steps.
To follow the example, refer to the timeline representation of steps and the table of calculated times (both below), and the definitions of student-step rollup fields. Note that steps alternately appear above and below the gray line to improve the readability of the example.
|Step #||Start Time||End Time||Step Duration (sec)||Correct Step Duration (sec)||Notes|
|11||15:32||15:42||10||null||A problem event precedes the first transaction for the step. DataShop uses the problem event time as the step start time. The step end time is the time of the last attempt on the step. No attempt is correct for this step, so the sum of the durations is the total length of time spent on the step, and there is no Correct Step Duration.|
|12||15:45||15:49||4||null||A problem event signifies a new instance of the same problem; it is used as the step start time. The correct attempt is not the first attempt, so again there is no Correct Step Duration.|
|2||null||46:00||null||null||No problem event precedes the first attempt for the step and the preceding transaction is more than 10 minutes before the first transaction on the step. Given this, DataShop does not calculate a step start time, nor a Step Duration or Correct Step Duration.|
|3||46:00||46:05||5||5||No problem event precedes the first attempt, but the preceding transaction's time is less than 10 minutes prior so it is used as the step start time. Correct Step Duration and Step Duration are equivalent because the first transaction is a correct attempt.|
|4||46:06||46:25||4+3+3=10||null||Step 4 is interrupted by attempts toward Step 5. DataShop excludes time spent toward Step 5 in its calculation of total time spent on Step 4. The step duration is the sum of the durations for transactions at 46:10 (4s), 46:13 (3s), and 46:25 (3s).|
|5||46:13||46:22||9||null||No problem event precedes the first attempt, but the preceding transaction's time is less than 10 minutes prior so it is used as the step start time.|