Primary Use Case Achievement

PSLC Primary Investigator 1

Distinguishing User Characteristic: User comes to the Data Shop with intimate knowledge of how his tutor works and the relationship between the tutor and the logs. User is currently collecting data with his tutor for a study.

Preexisting Condition: User has a username and password for the Data Shop.

Goal: Answer some research questions that he has using the Data Shop standard reports.

  1. User logs in to the Data Shop by entering his username and password in the text boxes and clicking "Login."
  2. Since user is listed as the PI for the project, user has immediate access to the data for his study. (Datasets are grouped by whether the user has permission to see them. Those you may use are listed at the top with the most recent study at the top.)
  3. User accesses his dataset by clicking the new link next to Analyses.
  4. User decides on a question that he wants to answer with the data, and navigates to the appropriate standard report in an attempt to answer that question. He uses the Settings panel on the left to bring up the main report.
  5. He notices something that looks interesting in the data (outlier, blip, spike, etc.) decides to explore it further by clicking it.
  6. User can see the detailed data that are contributing to the interesting phenomenon listed to the right of the main report.
  7. Using the inter-report navigation in the detail box at the right, the user drills in to one of the records in order to get a better idea of the story that is causing the interesting phenomenon.
  8. After reviewing a few more of these records, he decides there's an interesting story and clicks the Export this Data link at the top. It takes him to the Data Export tool with all of the appropriate data pre-loaded in the columns. He clicks Export and downloads all records that match the criteria.
  9. He performs additional statistical analyses using the statistical package and methods of his choice.

PSLC Primary Investigator 2

Distinguishing User Characteristic: User's study has finished and she wants to know the effectiveness of the treatment in the tutor.

Preexisting Condition: User is logged into the Data Shop.

Goal: She will need to show some of the preliminary results to her research group.

  1. User accesses the dataset for her experiment. (via the browse datasets screen)
  2. User is interested in seeing the learning curves for each skill in the curriculum covered by her tutor to see how her tutor stacked up against the control group. She uses the sample selector to group the students as appropriate into the treatment and control conditions. She names the groups and the group names appear in the sample selector window. She uses ctrl+click to select both of the groups. The learning curve report reveals the performance of each of the groups across the knowledge components.
  3. She browses the list of skill comparisons by slowly running her cursor over the list of knowledge components. Some of the skills she expected her tutor to teach better didn't do significantly better.
  4. She finds a comparison that she wants to show her research group, so she clicks Export Graph. A pop-up window appears that allows her to save it to her computer as a high-quality that can be used in her PowerPoint presentation.

PSLC Research Programmer

Distinguishing User Characteristic: User is developing tutor and currently running a pilot test to make sure there are no hidden skills in the tutor design.

Preexisting Condition: His dataset is a pilot study; this scenario is purely a debugging session. User is currently logged in.

Goal: Make sure two skills haven't been grouped into a single opportunity.

  1. User accesses the dataset for his pilot study via the browse datasets screen.
  2. He browses through the reports for the pilot study, and notices the extent of participation by observing the number of students on each of the reports.
  3. He accesses the learning curves for his data.
  4. The list of curves appears. He browses through the graphs by slowly running his cursor over the list. He is looking for a spike and finds one.
  5. User accesses the details on the curve with the spike by clicking the point. The contributing information is listed in the detail panel on the right.
  6. Since he's testing the skill decomposition of the problems he's written, all of the items are sequentially ordered. Attempt #6 is showing the spike.
  7. The point composition section shows that Problem #3 is causing the spike.
  8. He examines the overall performance on Problem #3 with the Problem Profile. He goes to the Error Reports, clicks the step analysis tab at the top and makes sure that Problem #3 is selected in the settings list. He clicks the error bar and looks at the Problem Info on the right to examine the error inputs for the problem.
  9. The research programmer launches his tutor, visits problem #3 and thinks about how to modify the problem.