<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="sun-etal-2020-clireval">
    <titleInfo>
        <title>CLIReval: Evaluating Machine Translation as a Cross-Lingual Information Retrieval Task</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Shuo</namePart>
        <namePart type="family">Sun</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Suzanna</namePart>
        <namePart type="family">Sia</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Kevin</namePart>
        <namePart type="family">Duh</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2020-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Asli</namePart>
            <namePart type="family">Celikyilmaz</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Tsung-Hsien</namePart>
            <namePart type="family">Wen</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Online</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>We present CLIReval, an easy-to-use toolkit for evaluating machine translation (MT) with the proxy task of cross-lingual information retrieval (CLIR). Contrary to what the project name might suggest, CLIReval does not actually require any annotated CLIR dataset. Instead, it automatically transforms translations and references used in MT evaluations into a synthetic CLIR dataset; it then sets up a standard search engine (Elasticsearch) and computes various information retrieval metrics (e.g., mean average precision) by treating the translations as documents to be retrieved. The idea is to gauge the quality of MT by its impact on the document translation approach to CLIR. As a case study, we run CLIReval on the “metrics shared task” of WMT2019; while this extrinsic metric is not intended to replace popular intrinsic metrics such as BLEU, results suggest CLIReval is competitive in many language pairs in terms of correlation to human judgments of quality. CLIReval is publicly available at https://github.com/ssun32/CLIReval.</abstract>
    <identifier type="citekey">sun-etal-2020-clireval</identifier>
    <identifier type="doi">10.18653/v1/2020.acl-demos.18</identifier>
    <location>
        <url>https://aclanthology.org/2020.acl-demos.18</url>
    </location>
    <part>
        <date>2020-07</date>
        <extent unit="page">
            <start>134</start>
            <end>141</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T CLIReval: Evaluating Machine Translation as a Cross-Lingual Information Retrieval Task
%A Sun, Shuo
%A Sia, Suzanna
%A Duh, Kevin
%Y Celikyilmaz, Asli
%Y Wen, Tsung-Hsien
%S Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations
%D 2020
%8 July
%I Association for Computational Linguistics
%C Online
%F sun-etal-2020-clireval
%X We present CLIReval, an easy-to-use toolkit for evaluating machine translation (MT) with the proxy task of cross-lingual information retrieval (CLIR). Contrary to what the project name might suggest, CLIReval does not actually require any annotated CLIR dataset. Instead, it automatically transforms translations and references used in MT evaluations into a synthetic CLIR dataset; it then sets up a standard search engine (Elasticsearch) and computes various information retrieval metrics (e.g., mean average precision) by treating the translations as documents to be retrieved. The idea is to gauge the quality of MT by its impact on the document translation approach to CLIR. As a case study, we run CLIReval on the “metrics shared task” of WMT2019; while this extrinsic metric is not intended to replace popular intrinsic metrics such as BLEU, results suggest CLIReval is competitive in many language pairs in terms of correlation to human judgments of quality. CLIReval is publicly available at https://github.com/ssun32/CLIReval.
%R 10.18653/v1/2020.acl-demos.18
%U https://aclanthology.org/2020.acl-demos.18
%U https://doi.org/10.18653/v1/2020.acl-demos.18
%P 134-141

Markdown (Informal)

[CLIReval: Evaluating Machine Translation as a Cross-Lingual Information Retrieval Task](https://aclanthology.org/2020.acl-demos.18) (Sun et al., ACL 2020)

• CLIReval: Evaluating Machine Translation as a Cross-Lingual Information Retrieval Task (Sun et al., ACL 2020)

ACL

• Shuo Sun, Suzanna Sia, and Kevin Duh. 2020. CLIReval: Evaluating Machine Translation as a Cross-Lingual Information Retrieval Task. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 134–141, Online. Association for Computational Linguistics.