<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="thawani-etal-2019-swow">
    <titleInfo>
        <title>SWOW-8500: Word Association task for Intrinsic Evaluation of Word Embeddings</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Avijit</namePart>
        <namePart type="family">Thawani</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Biplav</namePart>
        <namePart type="family">Srivastava</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Anil</namePart>
        <namePart type="family">Singh</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2019-06</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 3rd Workshop on Evaluating Vector Space Representations for NLP</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Anna</namePart>
            <namePart type="family">Rogers</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Aleksandr</namePart>
            <namePart type="family">Drozd</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Anna</namePart>
            <namePart type="family">Rumshisky</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yoav</namePart>
            <namePart type="family">Goldberg</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Minneapolis, USA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Downstream evaluation of pretrained word embeddings is expensive, more so for tasks where current state of the art models are very large architectures. Intrinsic evaluation using word similarity or analogy datasets, on the other hand, suffers from several disadvantages. We propose a novel intrinsic evaluation task employing large word association datasets (particularly the Small World of Words dataset). We observe correlations not just between performances on SWOW-8500 and previously proposed intrinsic tasks of word similarity prediction, but also with downstream tasks (eg. Text Classification and Natural Language Inference). Most importantly, we report better confidence intervals for scores on our word association task, with no fall in correlation with downstream performance.</abstract>
    <identifier type="citekey">thawani-etal-2019-swow</identifier>
    <identifier type="doi">10.18653/v1/W19-2006</identifier>
    <location>
        <url>https://aclanthology.org/W19-2006</url>
    </location>
    <part>
        <date>2019-06</date>
        <extent unit="page">
            <start>43</start>
            <end>51</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T SWOW-8500: Word Association task for Intrinsic Evaluation of Word Embeddings
%A Thawani, Avijit
%A Srivastava, Biplav
%A Singh, Anil
%Y Rogers, Anna
%Y Drozd, Aleksandr
%Y Rumshisky, Anna
%Y Goldberg, Yoav
%S Proceedings of the 3rd Workshop on Evaluating Vector Space Representations for NLP
%D 2019
%8 June
%I Association for Computational Linguistics
%C Minneapolis, USA
%F thawani-etal-2019-swow
%X Downstream evaluation of pretrained word embeddings is expensive, more so for tasks where current state of the art models are very large architectures. Intrinsic evaluation using word similarity or analogy datasets, on the other hand, suffers from several disadvantages. We propose a novel intrinsic evaluation task employing large word association datasets (particularly the Small World of Words dataset). We observe correlations not just between performances on SWOW-8500 and previously proposed intrinsic tasks of word similarity prediction, but also with downstream tasks (eg. Text Classification and Natural Language Inference). Most importantly, we report better confidence intervals for scores on our word association task, with no fall in correlation with downstream performance.
%R 10.18653/v1/W19-2006
%U https://aclanthology.org/W19-2006
%U https://doi.org/10.18653/v1/W19-2006
%P 43-51

Markdown (Informal)

[SWOW-8500: Word Association task for Intrinsic Evaluation of Word Embeddings](https://aclanthology.org/W19-2006) (Thawani et al., RepEval 2019)

• SWOW-8500: Word Association task for Intrinsic Evaluation of Word Embeddings (Thawani et al., RepEval 2019)

ACL

• Avijit Thawani, Biplav Srivastava, and Anil Singh. 2019. SWOW-8500: Word Association task for Intrinsic Evaluation of Word Embeddings. In Proceedings of the 3rd Workshop on Evaluating Vector Space Representations for NLP, pages 43–51, Minneapolis, USA. Association for Computational Linguistics.