Results > uq

Student Modelling with an Ensemble Approach

Overview

Supplementary online material

Provide a URL to a web page, technical memorandum, or a paper.

No response.

Background*

Provide a general summary with relevant background information: Where does the method come from? Is it novel? Name the prior art.

Our method represents a complex of three main directions: random forest in R, Naive Bayes (or averaging), and matrix factorisation, where different KC-opportunities were considered separately. Using above three models we produced 15 different solutions for the test set and for the randomly selected (labelled) subset of the training set, named S, which was set aside and was not used for the primary training. The size of the subset S was about 5% of the whole training data for the algebra and about 2% for the bridge_to_algebra sets. Then, using above 15 solutions as an explanatory variables (or features), we created a secondary training and test sets. The final ensemble solution was constructed by the GBM function in R. We are considering our method as a novel. During last 20 hours of the Challenge we observed a dramatic improvement, and we do not consider our results as a complete or as a final. Our method is a very flexible, and there are a lot of potentials for further improvements and developments.

Method

Summarize the algorithms you used in a way that those skilled in the art should understand what to do. Profile of your methods as follows:

Data exploration and understanding

Did you use data exploration techniques to

Please describe your data understanding efforts, and interesting observations:

No response.

Preprocessing

Feature generation

Details on feature generation:

No response.

Feature selection

Details on feature selection:

it was rather a feature smoothing based on the method of subintervals

Did you attempt to identify latent factors?

Details on latent factor discovery (techniques used, useful student/step features, how were the factors used, etc.):

also, we used gradient-based matrix factorisation

Other preprocessing

More details on preprocessing:

missings were treated as a special values

Classification

Base classifier

Loss Function

Regularizer

Ensemble Method

Were you able to use information present only in the training set?

Did you use post-training calibration to obtain accurate probabilities?

Did you make use of the development data sets for training?

Details on classification:

used k-means clustering applied to the latent factors with regularisation. The target of the regularisation is to ensure that all the clusters will be sufficiently large.

Model selection/hyperparameter selection

Details on model selection:

No response.

Results

Final Team Submission

Scores shown in the table below are Cup scores, not leaderboard scores. The difference between the two is described on the Evaluation page.

A reader should also know from reading the fact sheet what the strength of the method is.

Please comment about the following:

Quantitative advantages (e.g., compact feature subset, simplicity, computational advantages).

simplicity of the model is a very important when data are large and complex.

Qualitative advantages (e.g. compute posterior probabilities, theoretically motivated, has some elements of novelty).

We do believe that our method has quite interesting and novel theoretical grounds.

Other methods. List other methods you tried.

No response.

How helpful did you find the included KC models?

If you learned latent factors, how helpful were they?

Details on the relevance of the KC models and latent factors:

No response.

Software Implementation

Availability

Language

Details on software implementation:

R and Perl

Hardware implementation

Platform

Memory

Parallelism

Details on hardware implementation. Specify whether you provide a self contained-application or libraries.

No response.

Code URL

Provide a URL for the code (if available):

No response.

Competition Setup

From a performance point of view, the training set was

From a computational point of view, the training set was

Was the time constraint imposed by the challenge a difficulty or did you feel enough time to understand the data, prepare it, and train models?

How likely are you to keep working on this problem?

Comments on the problem (What aspects of the problem you found most interesting? Did it inspire you to develop new techniques?)

Yes, the problem is very interesting, and inspired us to develop a lot of new software (written in C). However, we had a lot of problems with pre-processings, which were conducted using Perl, and we can not be sure that all the problems have been resolved. We even requested the Organisers of the Cup (about one week before the deadline) to make an extension for one week.

References

List references below.

No response.