REPLAB 2014 @ CLEF 2014

Event Notification Type: 
Call for Participation
at CLEF 2014
Monday, 15 September 2014 to Thursday, 18 September 2014
United Kingdom
Submission Deadline: 
Saturday, 7 June 2014

REPLAB 2014 @ CLEF 2014


RepLab is a competitive evaluation exercise for Online Reputation Management systems. As in previous two years, the third RepLab campaign is an activity of CLEF, and the results of the exercise will be discussed at the CLEF 2014 conference in Sheffield, on the 15-18th September (see for details).

RepLab 2014 will focus on Reputation Monitoring on Twitter, targeting two new tasks: (1) the categorization of tweets with respect to standard reputation dimensions provided by the Reputation Institute ( and (2) the characterization of Twitter profiles related to a certain activity domain by classifying them by type (as journalists, professionals, etc.) and finding influential authors.

Participants are welcome to present systems that attempt one or both tasks (please refer to for details).

Note that Twitter profile classification forms part of the shared PAN-RepLab author profiling task. Besides the characterization of profiles from a reputation analysis perspective, participants can also attempt the classification of authors by gender and age, which is the focus of PAN 2014 (

In addition to the two tasks mentioned above, RepLab 2014 will include a track with peer-reviewed papers, TORM (Track for Online Reputation Monitoring) which will present research progress in tasks addressed by RepLab 2013 (,

For the reputation dimensions task, part of the RepLab 2013 data set will be used. It will contain around 70,000 tweets annotated according to their reputational dimension and belonging to the automotive and banking domains.

For the user profiling task, the data set consists of over 8,000 Twitter profiles (all with at least 1,000 followers) related to the automotive and banking domains. Each profile is represented by (i) profile name; (ii) profile URL and (iii) the last 600 tweets published by the author at crawling time. The opinion makers (i.e. authors with reputational influence) have been manually identified by reputation experts and are annotated as �Influencer�.

Each opinion maker is categorized as journalist, professional, authority, activist, investor, company or celebrity. The data set will be split into training and test subsets. The estimatated proportion is 30% and 70% respectively, although the exact splits will be given later.

Register for RepLab at the CLEF 2014 web page ( and contact the lab organizers ( and for further instructions.

March 1: Release of training data

March 17: Release of test data

May5: System results due

May 19: Official results released

June 7: Deadline for paper submissions

September 15-18: CLEF 2014 Conference in Sheffield

A call for papers for TORM (Track for Online Reputation Monitoring) will be released shortly.


- RepLab 2014 guidelines:

- CLEF conference web page:

- Facebook event (On this forum, you can share your impressions
and resolve doubts that might arise during the development):

RepLab is an activity sponsored by EU project LiMoSINe ( Lab organizers are:

Adolfo Corujo, Llorente & Cuenca

Julio Gonzalo, UNED

Maarten de Rijke, University of Amsterdam

Edgar Meij, Yahoo Labs

Enrique Amig�, UNED,

Jorge Carrillo de Albornoz, UNED

Damiano Spina, UNED

Irina Chugur, UNED

Vanessa �lvarez, Llorente & Cuenca

Ana Pitart, Llorente & Cuenca


Eugene Agichtein, Emory University, USA

Alexandra Balahur, JRC, Italy

Krisztian Balog, U. Stavanger, Norway

Donna Harman, NIST, USA

Eduard Hovy, ISI/USC, USA

Radu Jurca, Google, Switzerland

Jussi Karlgren, Gavagai/SICS, Sweden

Mounia Lalmas, Yahoo Research, Spain

Jochen Leidner, Thomson Reuters, Switzerland

Bing Liu, U. Illinois at Chicago, USA

Alessandro Moschitti, U. Trento, Italy

Miles Osborne, U. Edinburgh, UK

Hans Uszkoreit, U. Saarbrucken, Germany

James Shanahan, Boston U., USA

Belle Tseng, Apple, Inc.

Julio Villena, Daedalus/U. Carlos III, Spain