Apologies for multiple postings
+++++++++++++
CogALex-IV 'shared task'
concerning the ‘lexical access problem’
(Computing Associations when being given Multiple Stimuli)
In the framework of the 4th Workshop on Cognitive Aspects of the Lexicon (CogALex) to be held at COLING 2014, we invite participation in a shared task devoted to the problem of lexical access in language production, with the aim of providing a quantitative comparison between different systems.
MOTIVATION
The quality of a dictionary depends not only on coverage, but also on the accessibility of the information. That is, a crucial point is dictionary access. Access strategies vary with the task (text understanding vs. text production) and the knowledge available at the very moment of consultation (words, concepts, speech sounds). Unlike readers who look for meanings, writers start from them, searching for the corresponding words. While paper dictionaries are static, permitting only limited strategies for accessing information, their electronic counterparts promise dynamic, proactive search via multiple criteria (meaning, sound, related words) and via diverse access routes. Navigation takes place in a huge conceptual lexical space, and the results are displayable in a multitude of forms (e.g. as trees, as lists, as graphs, or sorted alphabetically, by topic, by frequency).
To bring some structure into this multitude of possibilities, the shared task will concentrate on a crucial subtask, namely multiword association. What we mean by this in the context of this workshop is the following. Suppose, we were looking for a word expressing the following ideas: 'superior dark coffee made of beans from Arabia', but could not remember the intended word 'mocha' due to the tip-of-the-tongue problem. Since people always remember something concerning the elusive word, it would be nice to have a system accepting this kind of input, to propose then a number of candidates for the target word. Given the above example, we might enter 'dark', 'coffee', 'beans', and 'Arabia', and the system would be supposed to come up with one or several associated words such as 'mocha', 'espresso', or 'cappuccino'.
TASK DEFINITION
The participants will receive lists of five given words (primes) such as 'circus', 'funny', 'nose', 'fool', and 'fun' and are supposed to compute the word which is most closely associated to all of them. In this case, the word 'clown' would be the expected response. Here are some more examples:
given words: gin, drink, scotch, bottle, soda
target word: whisky
given words: wheel, driver, bus, drive, lorry
target word: car
given words: neck, animal, zoo, long, tall
target word: giraffe
given words: holiday, work, sun, summer, abroad
target word: vacation
given words: home, garden, door, boat, chimney
target word: house
given words: blue, cloud, stars, night, high
target word: sky
We will provide a training set of 2000 sets of five input words (multiword stimuli), together with the expected target words (associative responses). The participants will have about five weeks to train their systems on this data. After the training phase, we will release a test set containing another 2000 sets of five input words, but without providing the expected target words.
Participants will have five days to run their systems on the test data, thereby predicting the target words. For each system, we will compare the results to the expected target words and compute an accuracy. The participants will be invited to submit a paper describing their approach and their results.
For the participating systems, we will distinguish two categories:
Unrestricted systems. They can use any kind of data to compute their results.
Restricted systems: These systems are only allowed to draw on the freely available ukWaC corpus in order to extract information on word associations. The ukWaC corpus comprises about 2 billion words and can be downloaded from http://wacky.sslmit.unibo.it/doku.php?id=corpora.
Participants are allowed to compete in either category or in both.
VENUE
The shared task will take place as part of the CogALex workshop which is co-located with COLING 2014 (Dublin). The workshop date is August 23, 2014. Shared task participants who wish to have a paper published in the workshop proceedings will be required to present their work at the workshop.
SHARED TASK SCHEDULE
Training data release: March 27, 2014
Test data release: May 5, 2014
Final results due: May 9, 2014
Deadline for paper submission: May 31, 2014
Reviewers' feedback: June, 15, 2014
Camera-ready version: July 7, 2014
Workshop date: August 23, 2014
FURTHER INFORMATION
CogALex workshop website: http://pageperso.lif.univ-mrs.fr/~michael.zock/CogALex-IV/cogalex-webpag...
Data releases: To be found on the above workshop website from the dates given in the schedule.
Registration for the shared task: Send e-mail to Reinhard Rapp, with Michael Zock in copy.
WORKSHOP ORGANIZERS
Michael Zock (LIF-CNRS, Marseille, France), michael.zock AT lif.univ-mrs.fr
Reinhard Rapp (University of Aix Marseille (France) and Mainz (Germany), reinhardrapp AT gmx.de
Chu-Ren Huang (The Hong Kong Polytechnic University, Hong Kong), churen.huang AT inet.polyu.edu.hk
--
------------------------------------------------
Michael ZOCK
Aix-Marseille Université,
CNRS & LIF, UMR 7279,
163 Avenue de Luminy
F-13288 Marseille / France
Mail: michael.zock [at] lif.univ-mrs.fr
Tel.: +33 (0) 4 91 82 94 88
Secr.: +33 (0) 4 91 82 90 70
Fax: +33 (0) 4 91 82 92 75
Web: http://pageperso.lif.univ-mrs.fr/~michael.zock/
------------------------------------------------
--------------------------------------------------------------------------------
From: Michael Zock
Date: April 25, 2014 at 1:10 AM
To: corpora [at] uib.no
CC: moses-support [at] mit.edu
Subject: Call for Participation in the "Shared task on the 'lexical access' problem (part of CogALex-IV, Coling, 2014)
Apologies for multiple postings
+++++++++++++
CogALex-IV 'shared task'
concerning the ‘lexical access problem’
(Computing Associations when being given Multiple Stimuli)
In the framework of the 4th Workshop on Cognitive Aspects of the Lexicon (CogALex) to be held at COLING 2014, we invite participation in a shared task devoted to the problem of lexical access in language production, with the aim of providing a quantitative comparison between different systems.
MOTIVATION
The quality of a dictionary depends not only on coverage, but also on the accessibility of the information. That is, a crucial point is dictionary access. Access strategies vary with the task (text understanding vs. text production) and the knowledge available at the very moment of consultation (words, concepts, speech sounds). Unlike readers who look for meanings, writers start from them, searching for the corresponding words. While paper dictionaries are static, permitting only limited strategies for accessing information, their electronic counterparts promise dynamic, proactive search via multiple criteria (meaning, sound, related words) and via diverse access routes. Navigation takes place in a huge conceptual lexical space, and the results are displayable in a multitude of forms (e.g. as trees, as lists, as graphs, or sorted alphabetically, by topic, by frequency).
To bring some structure into this multitude of possibilities, the shared task will concentrate on a crucial subtask, namely multiword association. What we mean by this in the context of this workshop is the following. Suppose, we were looking for a word expressing the following ideas: 'superior dark coffee made of beans from Arabia', but could not remember the intended word 'mocha' due to the tip-of-the-tongue problem. Since people always remember something concerning the elusive word, it would be nice to have a system accepting this kind of input, to propose then a number of candidates for the target word. Given the above example, we might enter 'dark', 'coffee', 'beans', and 'Arabia', and the system would be supposed to come up with one or several associated words such as 'mocha', 'espresso', or 'cappuccino'.
TASK DEFINITION
The participants will receive lists of five given words (primes) such as 'circus', 'funny', 'nose', 'fool', and 'fun' and are supposed to compute the word which is most closely associated to all of them. In this case, the word 'clown' would be the expected response. Here are some more examples:
given words: gin, drink, scotch, bottle, soda
target word: whisky
given words: wheel, driver, bus, drive, lorry
target word: car
given words: neck, animal, zoo, long, tall
target word: giraffe
given words: holiday, work, sun, summer, abroad
target word: vacation
given words: home, garden, door, boat, chimney
target word: house
given words: blue, cloud, stars, night, high
target word: sky
We will provide a training set of 2000 sets of five input words (multiword stimuli), together with the expected target words (associative responses). The participants will have about five weeks to train their systems on this data. After the training phase, we will release a test set containing another 2000 sets of five input words, but without providing the expected target words.
Participants will have five days to run their systems on the test data, thereby predicting the target words. For each system, we will compare the results to the expected target words and compute an accuracy. The participants will be invited to submit a paper describing their approach and their results.
For the participating systems, we will distinguish two categories:
Unrestricted systems. They can use any kind of data to compute their results.
Restricted systems: These systems are only allowed to draw on the freely available ukWaC corpus in order to extract information on word associations. The ukWaC corpus comprises about 2 billion words and can be downloaded from http://wacky.sslmit.unibo.it/doku.php?id=corpora.
Participants are allowed to compete in either category or in both.
VENUE
The shared task will take place as part of the CogALex workshop which is co-located with COLING 2014 (Dublin). The workshop date is August 23, 2014. Shared task participants who wish to have a paper published in the workshop proceedings will be required to present their work at the workshop.
SHARED TASK SCHEDULE
Training data release: March 27, 2014
Test data release: May 5, 2014
Final results due: May 9, 2014
Deadline for paper submission: May 31, 2014
Reviewers' feedback: June, 15, 2014
Camera-ready version: July 7, 2014
Workshop date: August 23, 2014
FURTHER INFORMATION
CogALex workshop website: http://pageperso.lif.univ-mrs.fr/~michael.zock/CogALex-IV/cogalex-webpag...
Data releases: To be found on the above workshop website from the dates given in the schedule.
Registration for the shared task: Send e-mail to Reinhard Rapp, with Michael Zock in copy.
WORKSHOP ORGANIZERS
Michael Zock (LIF-CNRS, Marseille, France), michael.zock AT lif.univ-mrs.fr
Reinhard Rapp (University of Aix Marseille (France) and Mainz (Germany), reinhardrapp AT gmx.de
Chu-Ren Huang (The Hong Kong Polytechnic University, Hong Kong), churen.huang AT inet.polyu.edu.hk