Leveraging the Wisdom of the Crowds for the Acquisition of Multilingual Language Resources

Arno Scharl, Marta Sabou, Stefan Gindl, Walter Rafelsberger, Albert Weichselbraun


Abstract
Games with a purpose are an increasingly popular mechanism for leveraging the wisdom of the crowds to address tasks which are trivial for humans but still not solvable by computer algorithms in a satisfying manner. As a novel mechanism for structuring human-computer interactions, a key challenge when creating them is motivating users to participate while generating useful and unbiased results. This paper focuses on important design choices and success factors of effective games with a purpose. Our findings are based on lessons learned while developing and deploying Sentiment Quiz, a crowdsourcing application for creating sentiment lexicons (an essential component of most sentiment detection algorithms). We describe the goals and structure of the game, the underlying application framework, the sentiment lexicons gathered through crowdsourcing, as well as a novel approach to automatically extend the lexicons by means of a bootstrapping process. Such an automated extension further increases the efficiency of the acquisition process by limiting the number of terms that need to be gathered from the game participants.
Anthology ID:
L12-1066
Volume:
Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12)
Month:
May
Year:
2012
Address:
Istanbul, Turkey
Editors:
Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Mehmet Uğur Doğan, Bente Maegaard, Joseph Mariani, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association (ELRA)
Note:
Pages:
379–383
Language:
URL:
http://www.lrec-conf.org/proceedings/lrec2012/pdf/210_Paper.pdf
DOI:
Bibkey:
Cite (ACL):
Arno Scharl, Marta Sabou, Stefan Gindl, Walter Rafelsberger, and Albert Weichselbraun. 2012. Leveraging the Wisdom of the Crowds for the Acquisition of Multilingual Language Resources. In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12), pages 379–383, Istanbul, Turkey. European Language Resources Association (ELRA).
Cite (Informal):
Leveraging the Wisdom of the Crowds for the Acquisition of Multilingual Language Resources (Scharl et al., LREC 2012)
Copy Citation:
PDF:
http://www.lrec-conf.org/proceedings/lrec2012/pdf/210_Paper.pdf