09:00 – 10:30 Introduction.

Keynote: “Offline evaluation of recommender systems: all pain and no gain?” [slides] – Mark Levy

10:30 – 11:00 Coffee Break
11:00 – 12:00 Paper session 1.

  • How to improve the statistical power of the 10-fold cross validation scheme in Recommender Systems.
    Andrej Košir, Ante Odić and Marko Tkalčič.
  • Research Paper Recommender System Evaluation: A Quantitative Literature Survey.
    Joeran Beel, Stefan Langer, Marcel Genzmehr, Bela Gipp, Corinna Breitinger, and Andreas Nürnberger.
12:00 – 14:00 Lunch
14:00 – 15:00 Paper session 2.

  • A Comparative Analysis of Offline and Online Evaluations and Discussion of Research Paper Recommender System Evaluation.
    Joeran Beel, Stefan Langer, Bela Gipp, Marcel Genzmehr, and Andreas Nürnberger.
  • Toward Identification and Adoption of Best Practices in Algorithmic Recommender Systems Research.
    Joseph Konstan and Gediminas Adomavicius.
15:00 – 16:00 Panel on “Reproducibility and replication in recommender systems evaluation”
16:00 – 16:15 Wrap-up