<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="suter-etal-2021-grounding">
    <titleInfo>
        <title>Grounding Plural Phrases: Countering Evaluation Biases by Individuation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Julia</namePart>
        <namePart type="family">Suter</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Letitia</namePart>
        <namePart type="family">Parcalabescu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Anette</namePart>
        <namePart type="family">Frank</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2021-06</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Second Workshop on Advances in Language and Vision Research</title>
        </titleInfo>
        <name>
            <namePart>Xin</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Ronghang</namePart>
            <namePart type="family">Hu</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Drew</namePart>
            <namePart type="family">Hudson</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Tsu-Jui</namePart>
            <namePart type="family">Fu</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Marcus</namePart>
            <namePart type="family">Rohrbach</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Daniel</namePart>
            <namePart type="family">Fried</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Online</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Phrase grounding (PG) is a multimodal task that grounds language in images. PG systems are evaluated on well-known benchmarks, using Intersection over Union (IoU) as evaluation metric. This work highlights a disconcerting bias in the evaluation of grounded plural phrases, which arises from representing sets of objects as a union box covering all component bounding boxes, in conjunction with the IoU metric. We detect, analyze and quantify an evaluation bias in the grounding of plural phrases and define a novel metric, c-IoU, based on a union box’s component boxes. We experimentally show that our new metric greatly alleviates this bias and recommend using it for fairer evaluation of plural phrases in PG tasks.</abstract>
    <identifier type="citekey">suter-etal-2021-grounding</identifier>
    <identifier type="doi">10.18653/v1/2021.alvr-1.4</identifier>
    <location>
        <url>https://aclanthology.org/2021.alvr-1.4</url>
    </location>
    <part>
        <date>2021-06</date>
        <extent unit="page">
            <start>22</start>
            <end>28</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Grounding Plural Phrases: Countering Evaluation Biases by Individuation
%A Suter, Julia
%A Parcalabescu, Letitia
%A Frank, Anette
%Y Hu, Ronghang
%Y Hudson, Drew
%Y Fu, Tsu-Jui
%Y Rohrbach, Marcus
%Y Fried, Daniel
%E Xin
%S Proceedings of the Second Workshop on Advances in Language and Vision Research
%D 2021
%8 June
%I Association for Computational Linguistics
%C Online
%F suter-etal-2021-grounding
%X Phrase grounding (PG) is a multimodal task that grounds language in images. PG systems are evaluated on well-known benchmarks, using Intersection over Union (IoU) as evaluation metric. This work highlights a disconcerting bias in the evaluation of grounded plural phrases, which arises from representing sets of objects as a union box covering all component bounding boxes, in conjunction with the IoU metric. We detect, analyze and quantify an evaluation bias in the grounding of plural phrases and define a novel metric, c-IoU, based on a union box’s component boxes. We experimentally show that our new metric greatly alleviates this bias and recommend using it for fairer evaluation of plural phrases in PG tasks.
%R 10.18653/v1/2021.alvr-1.4
%U https://aclanthology.org/2021.alvr-1.4
%U https://doi.org/10.18653/v1/2021.alvr-1.4
%P 22-28

Markdown (Informal)

[Grounding Plural Phrases: Countering Evaluation Biases by Individuation](https://aclanthology.org/2021.alvr-1.4) (Suter et al., ALVR 2021)

• Grounding Plural Phrases: Countering Evaluation Biases by Individuation (Suter et al., ALVR 2021)

ACL

• Julia Suter, Letitia Parcalabescu, and Anette Frank. 2021. Grounding Plural Phrases: Countering Evaluation Biases by Individuation. In Proceedings of the Second Workshop on Advances in Language and Vision Research, pages 22–28, Online. Association for Computational Linguistics.