<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="cohen-widdows-2018-bringing">
    <titleInfo>
        <title>Bringing Order to Neural Word Embeddings with Embeddings Augmented by Random Permutations (EARP)</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Trevor</namePart>
        <namePart type="family">Cohen</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Dominic</namePart>
        <namePart type="family">Widdows</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2018-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 22nd Conference on Computational Natural Language Learning</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Anna</namePart>
            <namePart type="family">Korhonen</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Ivan</namePart>
            <namePart type="family">Titov</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Brussels, Belgium</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Word order is clearly a vital part of human language, but it has been used comparatively lightly in distributional vector models. This paper presents a new method for incorporating word order information into word vector embedding models by combining the benefits of permutation-based order encoding with the more recent method of skip-gram with negative sampling. The new method introduced here is called Embeddings Augmented by Random Permutations (EARP). It operates by applying permutations to the coordinates of context vector representations during the process of training. Results show an 8% improvement in accuracy on the challenging Bigger Analogy Test Set, and smaller but consistent improvements on other analogy reference sets. These findings demonstrate the importance of order-based information in analogical retrieval tasks, and the utility of random permutations as a means to augment neural embeddings.</abstract>
    <identifier type="citekey">cohen-widdows-2018-bringing</identifier>
    <identifier type="doi">10.18653/v1/K18-1045</identifier>
    <location>
        <url>https://aclanthology.org/K18-1045</url>
    </location>
    <part>
        <date>2018-10</date>
        <extent unit="page">
            <start>465</start>
            <end>475</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Bringing Order to Neural Word Embeddings with Embeddings Augmented by Random Permutations (EARP)
%A Cohen, Trevor
%A Widdows, Dominic
%Y Korhonen, Anna
%Y Titov, Ivan
%S Proceedings of the 22nd Conference on Computational Natural Language Learning
%D 2018
%8 October
%I Association for Computational Linguistics
%C Brussels, Belgium
%F cohen-widdows-2018-bringing
%X Word order is clearly a vital part of human language, but it has been used comparatively lightly in distributional vector models. This paper presents a new method for incorporating word order information into word vector embedding models by combining the benefits of permutation-based order encoding with the more recent method of skip-gram with negative sampling. The new method introduced here is called Embeddings Augmented by Random Permutations (EARP). It operates by applying permutations to the coordinates of context vector representations during the process of training. Results show an 8% improvement in accuracy on the challenging Bigger Analogy Test Set, and smaller but consistent improvements on other analogy reference sets. These findings demonstrate the importance of order-based information in analogical retrieval tasks, and the utility of random permutations as a means to augment neural embeddings.
%R 10.18653/v1/K18-1045
%U https://aclanthology.org/K18-1045
%U https://doi.org/10.18653/v1/K18-1045
%P 465-475

Markdown (Informal)

[Bringing Order to Neural Word Embeddings with Embeddings Augmented by Random Permutations (EARP)](https://aclanthology.org/K18-1045) (Cohen & Widdows, CoNLL 2018)

• Bringing Order to Neural Word Embeddings with Embeddings Augmented by Random Permutations (EARP) (Cohen & Widdows, CoNLL 2018)

ACL

• Trevor Cohen and Dominic Widdows. 2018. Bringing Order to Neural Word Embeddings with Embeddings Augmented by Random Permutations (EARP). In Proceedings of the 22nd Conference on Computational Natural Language Learning, pages 465–475, Brussels, Belgium. Association for Computational Linguistics.