Sample Selector

Sample Selector is a tool for creating and editing samples, or groups of data you compare across—they're not "samples" in the statistical sense, but more like filters.

By default, a single sample exists: "All Data". With the Sample Selector, you can create new samples to organize your data.

You can use samples to:

A sample is composed of one or more filters, specific conditions that narrow down your sample.

Creating a sample

The general process for creating a sample is to:

The effect of multiple filters

DataShop interprets each filter after the first as an additional restriction on the data that is included in the sample. This is also known as a logical "AND". You can see the results of multiple filters in the sample preview as soon as all filters are "saved".

Help

Error Report

To simplify analysis, the Error Report examines data from student first attempts only.

In the Error Report, you can view information aggregated by problem step or knowledge component.

The Error Report provides:

  • summaries of student performance by step or knowledge component
  • actual values students entered and the feedback they received
  • at-a-glance information on problem coverage—the number of students exposed to a particular step or knowledge component, the knowledge components associated with each step, and the the problems that contain each knowledge component.

Viewing by problem

When viewing by problem, you can view only one problem at a time. Change the problem by clicking a problem's name from the scrollable list in the navigation sidebar.

The current problem is broken down by step, and lists the various attempts students made.

At the top of the page, the full problem name is shown.

Steps

A problem is commonly broken down into multiple steps. For each step, a list of details about that step are displayed.

Step Name
The first line of step information is the step name. The format of the step name varies across tutor technologies and logging implementations. CTAT tutors, for example, characterize step name as the widget the student interacted with (ie, the 'selection') and the action the student performed on that widget. (Student input—the third component of the selection-action-input format—is not displayed in this portion of the table since it varies by student response. Instead, if it exists, it appears under the Answer column of each row in the Attempts table.)
Number of Students
The total number of students that attempted this step of the problem.
Knowledge Component(s)
The knowledge component(s) that are associated with this step of the problem.
Sample
The name of the sample(s) from which the shown data comes. The sample name here will one of the selected samples shown in the upper-left corner of the sidebar.

Attempts

Evaluation types

Figure 1: Evaluation Types

Evaluation
Student attempts are classified into three evaluation types: correct, incorrect, and hint. Each evaluation type corresponds with a background color (see Figure 1).
Number of Students
The number of students that attempted this step. Expressed in the form: count (percentage of students that attempted this step).
Answer
The answer that the student provided on his or her first attempt for this step
Feedback
The feedback message provided by the tutor to the student.
Classification
Used to further describe the student-problem interaction (e.g., some tutors employ multiple hint and error levels).

Viewing by knowledge component

When viewing by knowledge component, you can select multiple knowledge components, but each will be shown in its own table row.

Values shown are aggregates by knowledge component. From these you can tell, for instance, the more difficult knowledge components by examining the percentages on corrects, errors, and hints.

Knowledge Component

Knowledge Component (KC) Name
Displayed in bold type.
Number of Observations
The total number of times a student took an opportunity to demonstrate the knowledge component.
Problem(s)
The problem(s) that contain steps with this knowledge component.
Sample
The name of the sample(s) from which the shown data comes. The sample name here will one of the selected samples shown in the upper-left corner of the sidebar.

Aggregate Values

Evaluation
Student attempts are classified into three evaluation types: correct, incorrect, and hint. Each evaluation type corresponds with a background color (see Figure 1).
Number of Observations
The number of times a student took an opportunity to demonstrate the knowledge component, broken down by evaluation. Expressed in the form: count (percentage of observations for this KC).

Error Report example