Background

Purpose of the Data Shop

The Data Shop Web Application, is intended for use by learning researchers in the Pittsburgh Science of Learning Center (PSLC). Two essential services that the Data Shop will provide are (a) a central repository of analysis data and (b) a standard reporting mechanism. The central repository of analysis data will be created from raw data collected from the seven PSLC LearnLab curricula: Algebra, Geometry, Chemistry, Physics, Chinese, French and English as a Second Language (ESL).

The goal of the PSLC is to make the analysis tool available to all participating scientists and developers. Over time, we intend to cover as many standard report types as possible, allowing researchers to perform a significant amount of their analyses within the tool. However, the Data Export tool within the Data Shop will provide the capability to specify and extract data so that more complex analyses can be performed in a variety of external statistical packages.

Goals of the Data Shop

Current Research Support

The Data Shop web application aims to allow the researcher (or members of the researcher's team) to log in, view their study data through standard reports, drill down into the more detailed information that makes up the standard reports and export the data or a desired sub-set. Researchers can then work with the data in their own computing environments using Excel or another statistics package of their choice.

Additionally, an API will be provided to researchers to help them populate the Transaction table (attempt/result pairs) as well as supporting tables in a Data Shop-type database. The API will allow users to write their own java programs in their own computing environments, to"distill" their raw data in whatever format it is currently in. Once their current data is in the Transaction table, researchers can perform further analyses using java, or by extracting their data from their local database to do further analysis.

Contextual inquiry research revealed that researchers have a need to quickly get a feel for the data and the stories it holds. Researchers are better supported when they can do these initial exploratory steps without interruption. "Being able to look at your results before you start doing analyses is very powerful" (CI5, 11:00). The Data Shop aims to support this initial exploration via standard reports and capabilities that allow researchers to closely examine and quickly understand detailed data. Additionally, the Data Shop will make it easy for researchers to download the data as needed to perform more detailed analyses in their own computing environments.

The MHCI team gathered data in the course of their work with users to assess the most commonly used file formats for the data export tools, however no clear preference in external analysis tools emerged. Initially, the ability to export to Excel will be practical for all users. The ability to export to a database format (SQL, mySQL) and to more complex statistical packages, primarily SPSS, will be beneficial for many users.

Additional Future Reports

As learning researchers identify reports that the entire PSLC community may find useful, contextual inquiries and requirements solicitation interviews should be performed to develop the specifications required to include the new reports into one of the web application's standard reports. These reports, in addition to those initially identified by the 2005 MHCI team (See Requirements and Spring Presentation), will become available in future releases of the web application. The current version will include a mechanism to request an additional report or provide feedback on the usefulness of the current application.

Encouraging Collaboration

The contextual inquiries also showed that researchers would like to collaborate but do not actively seek new collaborations for their own sake. It is hypothesized that lowering the search costs for identifying an interesting study or identifying a researcher doing related work will lead to collaboration opportunities.

The Data Shop Web Application will allow researchers to search and browse a virtual "table of contents," or a list of all relevant studies based on the search criteria. The curriculum description for each study will contain elements that will provide an overview of the original study, including elements that may be useful to spur collaboration, such as the study's principle investigator information, and links to the papers that were published based on the particular dataset, plus necessary information to cite the dataset in a new paper will also be provided. Finally, an access point to the raw data (Data Explorer) will be readily visible (provided the user has the appropriate permissions). The initial list of potentially relevant information is as follows:

When data sets are available to more than the original research team, the web application will also enable users to contact the researchers or data custodians via email to ask them about particular studies or results. This will be achieved through linking to the researchers' bios on the faculty profile page of the www.learnlab.org website. PI's or data custodians will also be able to review which users have downloaded their data.

For PI's to be able to communicate the intricacies of the tutors used to create the data and for people to work collaboratively on the same data, users should be able to place global (viewed by all) annotations on datasets.

Attracting New Users / Marketing (future)

The Data Shop can be a useful sales tool for recruiting new members to the PSLC. It should permit guest users to "test drive" the Data Shop capabilities on a dummy dataset. It should also provide a guided tour or tutorial for new or non-members to understand the range of features and reports available. To make it easy for a potential member to start the process of joining the PSLC, the web site should provide a form or contact information for a guest user to request to become a member.