Cro36WSD: A Lexical Sample for Croatian Word Sense Disambiguation

Domagoj Alagić, Jan Šnajder


Abstract
We introduce Cro36WSD, a freely-available medium-sized lexical sample for Croatian word sense disambiguation (WSD).Cro36WSD comprises 36 words: 12 adjectives, 12 nouns, and 12 verbs, balanced across both frequency bands and polysemy levels. We adopt the multi-label annotation scheme in the hope of lessening the drawbacks of discrete sense inventories and obtaining more realistic annotations from human experts. Sense-annotated data is collected through multiple annotation rounds to ensure high-quality annotations: with a 115 person-hours effort we reached an inter-annotator agreement score of 0.877. We analyze the obtained data and perform a correlation analysis between several relevant variables, including word frequency, number of senses, sense distribution skewness, average annotation time, and the observed inter-annotator agreement (IAA). Using the obtained data, we compile multi- and single-labeled dataset variants using different label aggregation schemes. Finally, we evaluate three different baseline WSD models on both dataset variants and report on the insights gained. We make both dataset variants freely available.
Anthology ID:
L16-1267
Volume:
Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)
Month:
May
Year:
2016
Address:
Portorož, Slovenia
Editors:
Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Sara Goggi, Marko Grobelnik, Bente Maegaard, Joseph Mariani, Helene Mazo, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association (ELRA)
Note:
Pages:
1689–1694
Language:
URL:
https://aclanthology.org/L16-1267
DOI:
Bibkey:
Cite (ACL):
Domagoj Alagić and Jan Šnajder. 2016. Cro36WSD: A Lexical Sample for Croatian Word Sense Disambiguation. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16), pages 1689–1694, Portorož, Slovenia. European Language Resources Association (ELRA).
Cite (Informal):
Cro36WSD: A Lexical Sample for Croatian Word Sense Disambiguation (Alagić & Šnajder, LREC 2016)
Copy Citation:
PDF:
https://aclanthology.org/L16-1267.pdf