This is a supplementary website for the tutorial on Replication of Recommender Systems Research held at the ACM RecSys Summer School.
Before attending the tuturial, make sure to prepare the following:
- Download and familiarize yourself with Lenskit, Librec, or RankSys.
- Download and extract the Movielens 100k dataset.
During the hand-on session, you will be expected to be able to execute and evaluate a recommendation using your selected tool.
- ControlledEvaluation performs a transparent, standalone, evaluation of RankSys, LensKit, and Mahout using RiVal. The evaluation metrics presented are fully comparable as the data splitting, candidate item generation, and metric calculation is done identically for all three cases.
- RankSysEvaluation performs the same type of evaluation, using RankSys’s internal evaluation methods. The evaluation metrics are only comparable with other evaluation made within RankSys with the same settings.
To build the project, do the following:
cd rsss2017/project mvn clean install
The code consists of two classes: RankSysEvaluation.java one which runs RankSys’ knn recommender using Movielens100k and evaluates the generated recommendations using RankSys internal evaluations methods, and ControlledEvaluation.java which runs RankSys and Mahout and evaluates the generated recommendations using RiVal.
To execute the RankSysEvaluation class, execute the following commands:
mvn exec:java -Dexec.mainClass="net.recommenders.rsss2017.RankSysEvaluation"
To execute the ControlledEvaluation class, execute the following commands:
mvn exec:java -Dexec.mainClass="net.recommenders.rsss2017.ControlledEvaluation"