Results
During the course of the project, we tested paper and flash prototypes of the Learning Curve, Problem Profile, Error Report and Sample Selector/Data Export features of the Data Shop. 37 users were tested, resulting in over 200 Usability Action Reports (UARs).
These UARs were then sorted using affinity diagramming, through which we discovered the following themes:Download sorted list of UARs (.xls)
General Issues
Help Functionality
Legends needed for color-coding in reports
Context Matters
Need for Condition property
User Preference for 'Data Explorer' (previous title for Data Export)
Step vs Subgoal label issues
Download all UARs for this feature (.xls)
Sample Selector
Early Design: link to SS goes unnoticed
Selecting Criteria
Filters: Don't show too much or too little
Transitioning between queries
Boolean Operators
Direct Manipulation
Download all UARs for this feature (.xls)
Inter-report Navigation
Learning Curves -> Error Reports
Learning Curves -> Problem Profiles
General Navigation Concerns
Download all UARs for this feature (.xls)
Learning Curves
Confusion about the Skill Model
Regression equation and other statistics
Roll-over curve previewer ("scrubbing the curves")
Desire for an opportunity cut-off
Opportunity points and affording clickability
Knowledge Component Listbox width
multi-selection in Knowledge Component ListBox
Confusion about 'All Components' label
Comparing Curves
Confusion about the Knowledge Component labels
Issues with the Point Composition Report
Download all UARs for this feature (.xls)
Problem Profiles
Error Rates Require Units
Good Aspects of Prob. Prof. navigation
Download all UARs for this feature (.xls)
Error Reports
Bad Aspects of the First (Tabular) Error Report
Error Classification Issues
Are Hints Considered Errors?
Bad Aspects of later Error Reports (tabs go unnoticed)
Download all UARs for this feature (.xls)
Data Export
Selecting Criteria
Summary/Counts of Data
Sorting
Sequential Data
Filters: Don't show too much or too little
Transitioning between queries
Boolean Operators
Direct Manipulation
Download all UARs for this feature (.xls)