<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="chrupala-2019-symbolic">
    <titleInfo>
        <title>Symbolic Inductive Bias for Visually Grounded Learning of Spoken Language</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Grzegorz</namePart>
        <namePart type="family">Chrupała</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2019-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Anna</namePart>
            <namePart type="family">Korhonen</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">David</namePart>
            <namePart type="family">Traum</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Lluís</namePart>
            <namePart type="family">Màrquez</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Florence, Italy</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>A widespread approach to processing spoken language is to first automatically transcribe it into text. An alternative is to use an end-to-end approach: recent works have proposed to learn semantic embeddings of spoken language from images with spoken captions, without an intermediate transcription step. We propose to use multitask learning to exploit existing transcribed speech within the end-to-end setting. We describe a three-task architecture which combines the objectives of matching spoken captions with corresponding images, speech with text, and text with images. We show that the addition of the speech/text task leads to substantial performance improvements on image retrieval when compared to training the speech/image task in isolation. We conjecture that this is due to a strong inductive bias transcribed speech provides to the model, and offer supporting evidence for this.</abstract>
    <identifier type="citekey">chrupala-2019-symbolic</identifier>
    <identifier type="doi">10.18653/v1/P19-1647</identifier>
    <location>
        <url>https://aclanthology.org/P19-1647</url>
    </location>
    <part>
        <date>2019-07</date>
        <extent unit="page">
            <start>6452</start>
            <end>6462</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Symbolic Inductive Bias for Visually Grounded Learning of Spoken Language
%A Chrupała, Grzegorz
%Y Korhonen, Anna
%Y Traum, David
%Y Màrquez, Lluís
%S Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
%D 2019
%8 July
%I Association for Computational Linguistics
%C Florence, Italy
%F chrupala-2019-symbolic
%X A widespread approach to processing spoken language is to first automatically transcribe it into text. An alternative is to use an end-to-end approach: recent works have proposed to learn semantic embeddings of spoken language from images with spoken captions, without an intermediate transcription step. We propose to use multitask learning to exploit existing transcribed speech within the end-to-end setting. We describe a three-task architecture which combines the objectives of matching spoken captions with corresponding images, speech with text, and text with images. We show that the addition of the speech/text task leads to substantial performance improvements on image retrieval when compared to training the speech/image task in isolation. We conjecture that this is due to a strong inductive bias transcribed speech provides to the model, and offer supporting evidence for this.
%R 10.18653/v1/P19-1647
%U https://aclanthology.org/P19-1647
%U https://doi.org/10.18653/v1/P19-1647
%P 6452-6462

Markdown (Informal)

[Symbolic Inductive Bias for Visually Grounded Learning of Spoken Language](https://aclanthology.org/P19-1647) (Chrupała, ACL 2019)

• Symbolic Inductive Bias for Visually Grounded Learning of Spoken Language (Chrupała, ACL 2019)

ACL

• Grzegorz Chrupała. 2019. Symbolic Inductive Bias for Visually Grounded Learning of Spoken Language. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6452–6462, Florence, Italy. Association for Computational Linguistics.