NIST is pleased to announce the second Low Resource Human Language Technology Evaluation (LoReHLT17) to support the development of human language technology (HLT) that can provide rapid and effective response to emerging incidents where the language resources are very limited. The evaluation consists of three tasks:
- Machine Translation
- Entity Discovery and Linking
- Situation Frame (lightweight event extraction)
As with the first LoReHLT evaluation, participants may choose to participate in any or all of the above tasks.
In LoReHLT, the evaluation language is a surprise, and is announced at the start of the evaluation period. The evaluation centers on an emergency scenario, such as a tsunami or earthquake, where humanitarian assistance and disaster relief agencies need situational awareness support but have no NLP technology available to deploy. There is no training data in the evaluation language per se. Instead, participants are invited to pivot from other related languages, take advantage of language "universals", use transliteration, or attempt to quickly bootstrap training resources during the evaluation.
For year two, there will actually be TWO surprise evaluation languages released simultaneously. You can do one or both. There will be a dry-run in July and the format evaluation period is in August, 2017.
The LDC has made a number of resources available in "related languages", including Hungarian, Amharic, Uzbek, Vietnamese, Farsi and more. Participants are invited to use these resources as starting points to pivot to the surprise evaluation language.
For further information regarding the evaluation and schedule, please consult the evaluation plan which can be found at https://www.nist.gov/itl/iad/mig/lorehlt17-evaluations
We look forward to your participation LoReHLT17.