Fifty-three teams of researchers have registered to the competition, of which twenty-five teams (with members from five continents) have submitted a model before the deadline. The submissions can be classified into three main categories. The first class includes models that can be described as variants of BEAST (14 submissions). This class provided the best predictions. Thirteen of the 14 submissions were ranked in places 1-13, with MSDs ranging between 0.0089 and 0.0114. The second class involves models that do not try to approximate the underlying processes, and use machine learning and/or similar statistical approaches (4 submissions). Models of this class provided the best fit of the estimation studies. Yet they failed in predicting the competition data, and were ranked in the last places, with MSDs of 0.0211 or higher. The third class can be classified as "others". It includes models that some of them could be described as generalizations of expected utility theory and prospect theory (6 submissions). The performance of these models on predicting the competition data was mild. The best predictors of this class models received MSD of 0.0124 and 0.0127, and were ranked 14-15th in the competition.
Models ranking
The models are ranked based on their MSD in the competition set (the column marked bold)
Rank |
100*MSD Estimation |
100*MSD Prediction |
Description |
1 | 0.5866 | 0.8893 | variant of BEAST |
2 | 0.7000 | 0.9308 | variant of BEAST |
3 | 0.7798 | 0.9325 | variant of BEAST |
4 | 0.6804 | 0.9516 | variant of BEAST |
5 | 1.0350 | 0.9658 | variant of BEAST |
6 | 0.7768 | 0.9688 | variant of BEAST |
7 | 0.6984 | 0.9781 | variant of BEAST |
- | 0.7080 | 0.985 | BEAST (baseline model) |
8 | 0.7005 | 0.9965 | variant of BEAST |
9 | 0.6700 | 1.0336 | variant of BEAST |
10 | 0.9850 | 1.0535 | variant of BEAST |
11 | 0.6458 | 1.0631 | variant of BEAST |
12 | 1.1366 | 1.0750 | variant of BEAST |
13 | 0.9984 | 1.1438 | variant of BEAST |
14 | 0.7355 | 1.2376 | other |
15 | 0.6097 | 1.2690 | other |
16 | 1.8817 | 1.7043 | Variant of BEAST |
17 | 2.6557 | 1.7794 | other |
18 | 1.0598 | 1.8342 | other |
19 | 0.4081 | 1.8427 | Statistical methods |
20 | 0.9507 | 1.8433 | other |
21 | 0.4859 | 2.1158 | Statistical methods |
22 | 0.0019 | 2.6010 | Statistical methods |
23 | 0.8155 | 5.0146 | Statistical methods |
24 | 0.1996 | NA | Statistical methods |
25 | 3.6314 | NA | other |
Models Description and Code (with author's approval):
- CPC2015 winner: Cohen, D., & Cohen, O. (2015).
Cohen's Beast Model
Description | 618 KB | |
Code | 8 KB |