When do we laugh?

Studies on laughter in dialogue have proposed resolving what laughter is about by looking at what laughter follows or is adjacent to, even though this assumption has not been tested. Our paper investigates the sequential relation between the laughter and the laughable. We propose a semantic/pragmatic account in which laughter is treated as a gestural event anaphor referring to a laughable. The laughable is a described , metalinguistic or exophoric event which, upon appraisal, triggers a positive psychological shift in the laugher. We analysed a natural dialogue corpus of French and Chinese, and found that the time alignment between laughter and laughable is rather free. Only 30% of laugh-ters immediately follow the laughable. Laughter can occur (long) before, during, or (long) after the laughable; laughter overlapping with speech may not be about the co-occurring speech. Our results falsifies the assumption that what laughter follows is what it is about, and thus questions claims which rely on this assumption.


Introduction
Studies about laughter in interaction have been mainly focused on the acoustic or perceptual features, and often observations of the events preceding to it have been the base for claims concerning what laughter is about. (Provine, 1993) made a claim that has been subsequently adopted in much of the literature: laughter is, for the most part, not related to humour, because it is found to most frequently follow banal comments. Similar reasoning has been adopted by several other studies on the kind of situations that elicit laughter. The deduction process in these studies rely on an important yet untested assumption: what laughter follows is what it is about. Our paper investigates this assumption. We first briefly discuss previous studies on laughter in interaction; we then argue for a semantic/pragmatic account in which we treat laughter as a gestural event anaphora referring to a laughable. We present a corpus study of laughables and evaluate our results against previous proposals.

Studies on what laughter is about
In (Provine, 1993), the researcher observed natural conversations, and "when an observer heard laughter, she recorded in a notebook the comment immediately preceding the laughter and if the speaker and/or the audience laughed, the gender, and the estimated age of the speaker and the audience [...]. A laugh episode was defined as the occurrence of audible laughter and included any laughter by speaker or audience that followed within an estimated 1 s of the initial laugh event. The laugh episode included the last comment by a speaker if it occurred within an estimated 1 s preceding the onset of the initial laughter. A laugh episode was terminated if an estimated 1 s passed without speaker or audience laughter, or if either the speaker or the audience spoke.". They found that "Only about 10-20% of episodes were estimated by the observers to be humorous" (Provine, 1993), and thus derived the conclusion which is now widely adopted in the literature: laughter is, for the most part, not related to humour but about social interaction. An additional conclusion based on this study is that laughter never interrupts speech but "punctuates" it occurring exclusively at phrase boundaries.
Similarly, (Vettin and Todt, 2004) used exclusively timing parameters -i.e., what precedes and what follows the laugh (within a threshold of 3s)to distinguish 6 different contexts (see table 1) for laughter occurrence to support claims about situations that elicit laughter.

Weaknesses
In (Provine, 1993), the author assumed that laughter always immediately follows the laughable. Not only do the methods described above provide imprecise data (timing information was estimated during observation), it prevents the possibility of recording any data where laughter does not follow the laughable. In addition, even when the com-Conversational Parter A participant's laughter occurring immediately (up to 3s) after a complete utterance of their conversational partner Participant The participant laughed immediately (up to 3s) after his/her own complete utterance Short confirmation Participan's laughter immediately (up to 3s) after a confirming 'mm', 'I see' or something comparable by himself or his conversational partner Laughter Participant's laughter after a conversational partner's laughter. With an interval of less than 3s Before utterance Participant's laughter after a short pause (at least 3s) in conversation, but immediately (up to 500ms) before an utterance by him/herself Situation Laughter occurring during a pause in conversation (at least 3s), not followed by any utterance. The laughter is attibuted to the general situation and not to an utterance Table 1: Vetting and Todt, 2004 -Context classification ment that immediately precedes laughter is the actual trigger for a laugh, and it is not "amusing" in itself (i.e. it is a "banal comment"), it doesn't necessarily entail that the laughable is not humourous. The funniness might arise from the "banal comment" in relation to the previous utterance, the context of the interaction, shared experiences between the speakers, world knowledge and cultural conventions. For example, in (1) "what's funny" resides in the implicit content that the utterance refers to. In (2), the preceding utterance is funny only in relation to the context.
(1) A: Do you remember that time? B and A: < laughter/ >. Laughable= the enriched denotation of 'that time'.
(2) (Context: the speakers are discussing the plan of an imagined shared apartment, and they have already planned two bathrooms). A: I want another bathroom. B: < laughter/ > Laughable= "I want another bathroom" (Vettin and Todt, 2004) is methodologically more precise than (Provine, 1993), and they allow for the possibility that in addition to laughter occurring after the laughable, a laughter may precede an utterance, or occur during an exophoric situation. However, this analysis excludes laughters that occur in the middle of or overlaps with an utterance, and it uses exclusively timing parameters to determine what laughter is about (as illustrated in figure 1). For example, whether a laugh is considered to be about the preceding utterance or about the following utterance is decided purely on the difference in the length of gaps with the two utterances. Crucially, the conclusion is also drawn assuming an adjacency relationship between laughter and laughable.

Laughter as an event anaphor
We argue that previous studies have ignored analysing the laughable because they did not attempt to integrate their account with an explicit semantic/pragmatic module on the basis of which content is computed. 1 The sole recent exception to this, as far as we are aware, is the account of (Ginzburg et al., 2015), which sketches an information state-based account of the meaning and use of laughter in dialogue.
Taking this as a starting point, we argue that laughter is a gestural event anaphor, whose meaning contains two dimensions: one dimension about the arousal and the other about the trigger or the laughable. In line with (Morreall, 1983) we think that laughter effects a "positive psychological shift", and the "arousal" dimension signals the amplitude in the shift. 2 . The positive psychological shift is triggered by an appraisal of an event -the laughable l, and the second dimension communicates the type of the appraisal. (Ginzburg et al., 2015) propose two basic types of meaning in the laughable dimension: the person laughing may express her perception of the laughable l as being incongruous, or just that l is enjoyable (playful). We propose that in addition, certain uses of laughter in dialogue may suggest the need for a third possible type: expressing that l is a socially close ingroup situation.

Formal treatment of laughter
Here we sketch a formal semantic and pragmatic treatment of laughter. On the approach developed in KoS (Ginzburg, 2012), information states comprise a private part and the dialogue gameboard that represents information arising from publicized interactions. In addition to tracking shared assumptions/visual space, Moves, and QUD, the dialogue gameboard also tracks topoi and enthymemes that conversational participants exploit during an interaction (e.g., in reasoning about rhetorical relations.). Here topoi represent general inferential patterns (e.g., given two routes choose 1 This is not the case for some theories of humour, e.g., that due to (Raskin, 1985), who offers a reasonably explicit account of incongruity emanating from verbal content without, however, attempting to offer a theory of laughter in conversation. 2 The amplitudes in the shift depend on both the trigger itself and on the individual current information/emotional state. It is important to point out that laughter does not signal that the speaker's current emotional state is positive, merely that there was a shift which was positive. The speaker could have a very negative baseline emotional state (being very sad or angry) but the recognition of the incongruity in the laughable or its enjoyment can provoke a positive shift (which could be very minor) The distinction between the overall emotional state and the direction of the shift explains why laughter can be produced when one is sad or angry. the shortest one) represented as functions from records to record types, and enthymemes are instances of topoi (e.g., given that the route via Walnut street is shorter than the route via Alma choose Walnut street). An enthymeme belongs to a topos if its domain type is a subtype of the domain type of the topos. (Ginzburg et al., 2015) posit distinct, though quite similar lexical entries for enjoyment and incongruous laughter. For reasons of space in (3) we exhibit a unified entry with two distinct contents.
(3) associates an enjoyment laugh with the laugher's judgement of a proposition whose situational component l is active as enjoyable; for incongruity, a laugh marks a proposition whose situational component l is active as incongruous, relative to the currently maximal enthymeme under discussion.
(3) makes appeal to a notion of an active situation. This pertains to the accessible situational antecedents of a laughter act, given that (Ginzburg et al., 2015) proposed viewing laughter as an event anaphor. However, given the existence of a significant amount of speech laughter, as we discuss below, this notion apparently needs to be rethought somewhat, viewing laughter in gestural terms. This requires interfacing the two channels, a problem we will not address here, though see (Rieser, 2015) for a recent discussion in the context of manual gesture. (3) spkr : Ind addr : Ind t : TIME c1 : addressing(spkr,addr,t) MaxEud = e : (Rec)RecType The dialogue gameboard parameters utilised in the account of (Ginzburg et al., 2015) are all 'informational' or utterance related ones. However, in order to deal with notions such as arousal and psychological shift, one needs to introduce also parameters that track appraisal (see e.g., (Scherer, 2009)). For current purposes, we mention merely one such parameter we dub pleasantness that relates to the appraisal issue-in Scherer's formulation-Is the event intrinsically pleasant or unpleasant?. We assume that this parameter is scalar in value, with positive and negative values corresponding to varying degrees of pleasantness or unpleasantness.
This enables us to formulate conversational rules of the form 'if A laughs and pleasantness is set to k, then reset pleasantness to k + θ(α)', where α is a parameter corresponding to arousal.

Research questions
The study is part of a broader project where we analyse laughter using a multi-layered scheme and propose a semantic/ pragmatic account of the meaning and effects of laughter. The focus of the current study is the positioning of laughter in relation to its laughable.
Our account suggests that resolving the laughable is crucial for deriving the content of a laughter event. We hypothesize that laughter is not always adjacent to its laughable. Rather, the sequential distribution between laughter and laughable is somewhat free, illustrated in Figure 2. We hypothesize that laughter can occur before, during and after the laughable, and that it is possible for intervening materials to occur between a laughter event and its laughable. In more detail, we make the following hypotheses in relation to our research questions: Q1: Does laughter always follow its laughable?
-If not, does laughter-laughable alignment differ among different types of laughters?
We hypothesize that laughter can occur before, during or after the laughable; laughter and laughable should not have a one-to-one relationship: one laughable can be the referent of several laughter events.
-More specifically, laughter-laughable alignment may vary depending on at least the source of the laughable (self or partner) and whether it is speech laugh or laughter bouts. Q2: Does laughter interrupt speech?
We hypothesize that laughter can occur both at utterance boundaries and at utterance-medial position.
Q3: Is laughter-laughable alignment pattern language specific?
We hypothesize that language/culture influence alignment and thus predict to find differences between, in this case, French and Chinese.
3 Material and method

Corpus
We analyzed a portion of the DUEL corpus (Hough et al., 2016a) The corpus consists of 30 dyads (10 per language)/ 24 hours of natural, faceto-face, loosely task-directed dialogue in French, Mandarin Chinese and German. Each dyad conversed in three tasks which in total lasted around 45 minutes. The three tasks used were: 1. Dream Apartment: the participants are told that they are to share a large open-plan apartment, and will receive a large amount of money to furnish and decorate it. They discuss the layout, furnishing and decoration decisions; 2. Film Script: The participants spend 15 minutes creating a scene for a film in which something embarrassing happens to the main character; 3. Border control: one participant plays the role of a traveller attempting to pass through the border control of an imagined country, and is interviewed by an officer. The traveller has a personal situation that disfavours him/her in this interview. The officer asks questions that are general as well as specific. In addition, the traveller happens to be a parent-in-law of the officer.
The corpus is transcribed in the target language and glossed in English. Disfluency, laughter, and exclamations are annotated. The current paper presents analysis of laughter in two dyads in French and Chinese (3 tasks x 2 pairs x 2 languages).

Audio-video coding of laughter
Coding was conducted by the first and second authors and by 2 trained, but naïve to the aim of the study, masters students: each video was observed until a laugh occurred. The coder detected the exact onset and offset in Praat (Boersma and others, 2002), and conducted a multi-layer analysis as explained shortly. A laugh was identified referring to the same criteria used in (Nwokah et al., 1994), based on the facial expression and vocalization descriptions of laughter elaborated by (Apte, 1985) and (Ekman and Friesen, 1975). Following (Urbain and Dutoit, 2011) we counted laughter offset (final laughter in-breath inhalation) as part of the laughter event itself, thus resulting in laughter timings longer than other authors (Bachorowski and Owren, 2001;Rothgänger et al., 1998). All laughter events were categorised according to different parameters: formal and contextual aspects, semantic meaning and functions (see Table  2). The formal and contextual level analysis include whether a laughter overlaps speech (speech laugh), whether it co-occurs with or immediately follows a partner's laughter (dyadic/ antiphonal laughter), and its position in relation to the laughable. The semantic meaning level analysis include perceived arousal and whether it contains an element of incongruity could be identified by the coders. The function analysis codes the effect of laughter on the interaction, and distinguishes whether the effect is cooperative, i.e., promotes interaction (e.g. showing enjoyment, smoothing) or non-cooperative, i.e., in some way disaffects interaction (e.g., mocking or evade questions). Due to space constraints and current focus, we do not provide a detailed explanation of the multi-level laughter coding scheme, for which see (Mazzocconi et al., 2016). Reliability was assessed by having a masters student as a second coder for 10% of the material observed. Percentage agreements between the two coders for French and Chinese data averaged respectively 87% and 87.76, with an overall Krippendorff α (Krippendorff, 2012) across all tiers of 0.672 and 0.636.
For the main analysis, we include in our analysis both laughter and speech laughter (Nwokah et al., 1999). In the current study we restrict our observations about the aspects pertaining to the form, to the contextual distribution and positioning of a laugh in relation to others' laughter, the laughable and laugher's herself speech.

Identifying laughables
We consider as the laughable the event which, after appraisal, produces a positive psychological shift in the laugher. We distinguish three different kinds of laughable types: described events, metalinguistic stimuli and exophoric events. We also mark whether they originated from the laugher him/herself or by the partner.

Audio-video coding of laughable
Every time a laugh was identified, coders would mark on the Praat TextGrid, based on personal inference, the laughable the laugh would refer to.
The time boundaries were marked, the content (whether verbal or not) was annotated and an index was assigned in order to map laughter (or multiple laughters) and laughable. Laughables were classified according to three main categories: described, metalinguistic and exhophoric event. Reliability of type assignement was assessed by having a masters student as a second coder for 10% of the material observed. Percentage agreements between the two coders for French and Chinese averaged 92.5% with a Krippendorff α (Krippendorff, 2012) of 0.77.

Results
In our data sample (summarized in Table2), laughter is very frequent, constituting 17% of the conversation duration in French and 7.2% in Chinese. Each laughable is "laughed about" more than once (1.7 times in French and 1.4 times in Chinese).  To investigate the time alignment between laughter and laughable, we calculated "start of laughter minus start of laughable", "end of laughter minus end of laughable", and "start of laughter minus end of laughable". If laughter always follow the laughable, all three measurements should be above zero. This was not the case. In both Chinese and French, on average, laughter starts during rather than after the laughable, and finishes af-ter the laughable. In general, laughs in Chinese are more likely to overlap with the laughable than in French. The distribution varies over a wide range. Table 3 summarizes the gaps between the boundaries of laughter and laughable, and figure 3 plots specifically the gap between the end of the laughable and the start of laughter. They show that it is common for laughs to start before, during and after the laughable. When a laugh has no overlap with its laughable, they are not always adjacent to each other (average utterance duration is under 2 seconds while the gap can be up to 10 seconds). In the following example, the first two instances of speech laugh refer to a laughable in a later utterance.
(Translation) B: The teacher asked Xiaoshi to make a sentence with "bu yue er tong" (coincidentally together). Xiaoshi < laughspeech > then < laughspeech/ > thought about it, and said, uh, < laughspeech > (I + Xiaoshi) < laughspeech/ > thought about it and said, uh, uh I saw a pretty girl in the street, and I asked her "shall we go for a date?", and < laughspeech > the girl said "shouldn't date children" < laughspeech/ >. (note: "shouldn't date children" is phonologically identical to "incidentally together") Laughable= "the girl said 'shouldn't date children' " Based on whether laughter occurs entirely outside or overlapping with the laughable, we grouped the laughters into 4 alignment categories: "before", "overlap", "immediately after" and "other after" (see figure 4). We found that in both languages, laughters that immediately follow (within 0.3s) the laughable constitute 30% . There are more overlapping laughters in Chinese than in French (χ 2 (1)=6.9, p= .008).  4.2 Does laughter-laughable alignment differ among different "types" of laughables and laughters?
Our analysis mainly focuses on the distinction between self and partner produced laughables, and between speech laugh and laughter bouts, presented separately below. Due to space constraints, the effect of the rest of the tiers are not discussed.

Self vs. partner produced laughables
We coded whether the laughables are described events, meta-linguistic, or exophoric events. In our corpus described events are the commonest (92% in French and 89% in Chinese), followed by exophoric laughables (7% in French and 10%). Metalinguistic (1% in both languages) laughables are rare, so we grouped them with described events in the current analysis. On average, there are more self-produced than partnerproduced laughables, supporting the idea that speakers laugh more often than the audience. Interestingly, 3% of the laughables are jointly produced (one person finishing the other's sentence, or both saying roughly the same thing at the same time) (see (8)). With the former two categories, we also coded whether the laughable is produced by the laugher or her partner, which allow us to compare our results with studies of "speaker" or "audience" laughter.
(8) (totally overlapping turns are italicized) B: c'est une personne qui est aux toilettes dans < laughter > des toilettes publiques A: < laughter > X ah: oui: oui un mec qui parle a cute' < laughter/ > B: dans < laughter > des toilettes publiques voila sauf que l'autre il est au telephone et l'autre il lui croit qu'il parle . C'est genant < laughter/ > (Translation) B: it is a person who is in the bathroom in < laughter > in public bathroom A:< laughter > Ah yes yes a guy who is talking in the next stall < laughter/ > B: in < laughter > in public bathroom exactly but the other is on the phone and the other thinks he is speaking with him. That's embarrassing < laughter/ > Laughable= "exactly but the other is on the phone and the other thinks he is speaking with him" We found that laughters about a partnerproduced laughable start later than those about a self-produced laughable, but still the average starting time is before the end of the laughable. With partner-produced laughables, the average gap between the end of laughable and start of laughter is -0.02s in French and -0.3s in Chinese, while with self-produced laughables, the average gap is -0.7s in French and -1.3s in Chinese.

Speech laugh vs. laughter bouts
Laughter frequently overlaps with speech. 36% of laughter events in French and 47% of laughter events in Chinese contain speech laughter. Speech laughter is on average 0.3 seconds longer than stand alone laughter bouts. Speech laughs overlap with the laughable more than laughter bouts. 52% of speech laughters in French and 70% in Chinese overlap with the laughables. In comparison, 33% of laughter bouts in French and 34% in Chinese overlap with the laughable. The reason why speech laugh more often overlap with the laughables is likely to do with the difference in function between speech laugh and laughter bouts. Laughters that mark an upcoming laughable most frequently overlaps with speech, and these laughter events are also ones that tend to stretch until the middle or the end of the laughable. A more detailed analysis of the function/effect of laughter is reported in (Mazzocconi et al., 2016).
Notice that not all speech laughs overlap with the laughable, suggesting that often, laughter that co-occurs with speech is not about the cooccurring speech (47.8% in French and 30% in Chinese). In the following example, speaker B says that she'll take the bigger bedroom, and laughs. Speaker A joins the laughter but starts a new utterance.

Does laughter interrupt speech?
We investigated whether laughter occurs at utterance-medial positions when one party is speaking, and when the partner is speaking. Does laughter interrupt partners' utterances? Yes. We found that 51.8% of laughter bouts in French and 56.7% of laughter bouts in Chinese start during the partner's utterances (not necessarily laughables), for example: (10) B: pour faire un mur de son quoi < laughspeech > en fait c'est une < english > ra:ve < /english > notre appartement < /laughspeech > A: < laughter/ > (Translation) B: to create a sound barrier which < laughspeech > in fact it is a rave, our apartment < /laughspeech > A:< laughter/ > Laughable= "in fact it is a rave, our apartment" Does laughter interrupt one's own utterances?
(Translation) Do you have, uh, have any < laughter/ > criminal records?

Discussion
The aim of the current study was to deepen the little research available on the relation between laughter, laughable and speech in natural conversation, starting from the observation of their temporal sequence and alignment. We investigated three questions: whether laughter always follows, or at least is adjacent to its laughable, as is commonly assumed; whether this sequential alignment differ depending on differeht "types" of laughters; and whether laughter always punctuates speech. Our main findings are: 1. Time alignment between laughter and laughable is rather free.
-Laughter and laughable does not have a one-to-one relationship. A laughable can be referred to by more than one laughters.
-Contrary to popular belief, only 30% of laughters occur immediately after the laughable. Laughters frequently start during the laughable (more so with "speaker" laughter than "audience" laughter).
-Laughters can occur long before or long after the laughable, and not be adjacent to their laughable.
-Between 30 to 50 percent of speech laughs do not overlap with the laughable, suggesting that frequently laughs are not about the cooccurring speech.
If looking just at laughter bouts, about 40% occur immediately after the laughable.
2. Laughter-laughable alignment may differ depending on the different "types" of laughable and laughter. Specifically, laughters about a partner-produced laughable (audience laughter) start later than those about a self-produced laughable (speaker laughter). Speech laughs occur earlier than laughter bouts, and overlaps more with the laughable.
3. Comparing Chinese and French, the majority of the patterns are similar, except that in Chinese, laughs are more likely to overlap with the laughable than in French. This provides an initial indication that while certain aspects of laughter behaviour are influenced by culture/language, generally we use laughter similarly in interaction. 3 4. Laughter does interrupt speech: we often laugh when others are speaking (half of all laughter bouts) and occasionally we insert stand-alone laughters mid-sentence (less than 10%). Moreover, very frequently laughter overlaps speech (around 40% of all laughters).
The relatively free alignment between laughter and speech seems analogous at a first approximation to the relation between manual gesture and speech (Rieser, 2015). We propose to consider laughter as a verbal gesture, having an independent channel from speech, with which it communicates through an interface.

Is laughter rarely about funny stimuli?
Our results discredit the method of inferring what the laughter is about by looking at the elements that immediately precede or follow it. Therefore, previous conclusions using this method should be revisited (Provine, 1993;Provine, 1996;Provine, 2001;Provine and Emmorey, 2006;Vettin and Todt, 2004). One such conclusion is that because they follow "banal comments", laughter is mostly about not about funny stimuli. We have shown that the logic does not hold, as very often, those preceding "banal comments" are not the laughables. And even if they are, the "funniness" or incongruity may reside between the laughable and something else, e.g., the context of occurrence, world knowledge, cultural norms, experiences, informational and intentional states shared between interlocutors. For example, in the following exchange, the exchange seems rather banal, but in fact, they are laughing about the exophoric situation that they are acting. Exactly what proportion of laughables contain funny incongruity is a topic for further research. For now, our results questions the validity of existing proposals on this score.

Laughter Punctuating Speech?
It has been suggested (notably by Provine) that laughter bouts almost never (0.1%) disrupt phrases but punctuate them (Provine, 1993;Provine, 1996;Provine, 2001). He explains this finding on the basis of an organic constraint: laughter and speech share the same vocal apparatus and speech has "priority access". Curiously enough, Provine has always excluded speech-laughs from his investigations, without any justification. A more recent study on laughter in deaf ASL signers (Provine and Emmorey, 2006) showed that signers rarely laugh during their own utterances, where no competition for the same channel of expression is present. Provine and Emmory conclude that the punctuation effect of laughter holds even for signers, and possibly is not a simple physical constraint that determines the placement of laughter in dialogues, but due to a higher order linguistic ordered structure (Provine, 2006).
On the surface, their findings in speakers and signers are similar: speakers do not stop midsentence to insert a laugh, and signers do not laugh while signing a sentence. However, this "similarity" may be a difference in disguise. We have shown that speakers frequently overlap laughter and speech. If it were indeed true that signers do not laugh while signing, it raises the question why speech laughter is common for speakers but rare for signers. (Provine and Emmory, 2006) hypothesised that the placement of laughter in dialogue is controlled by a higher linguistic ordered structure, where laughter is secondary to language. Therefore, even when the two don't occur in competing channels, e.g., for signers, laughter still only occurs at phrase boundaries.
We argue for a different explanation. Assuming speech laughter data (laughter that overlaps utterances) were not excluded in the ASL study as they were in spoken dialogue studies, in deaf signers, since the laughter is perceived only visually and involves marked facial movements, it would interfere with the perception of the message conveyed by language. In sign languages, body and face movements constitute important communicative elements at all linguistic levels from phonology to morphology, semantics, syntax and prosody (Liddell, 1978;Campbell, 1999). Despite the fact that emotional facial expressions can overlap with linguistic facial movements (Dachkovsky and Sandler, 2009), a laugh, implying a significant alteration of facial configuration (see identification of a laughter episode) could be excessively disruptive for the message aimed to be conveyed. While in verbal language the laughter signal can be completely fused in the speech as a paralinguistic feature (Crystal, 1976) and used in a sophisticated manner to enrich and facilitate communication, (Nwokah et al., 1999) report that not even from an acoustic perspective is laughter secondary to speech: when co-occurring the laugh indeed does not resemble the speech spectral patterns nor does the speech resemble the laughter ones, but together they create a new idiosyncratic pattern. Laughter is fully meaningful and communicative in itself, universally across cultures, and the emo-tional components that it carries are not secondary to speech or trivial.

Conclusion and future work
Our study provides the first systematic analysis of laughables, and demonstrates the existence of a corpus, the DUEL corpus (Hough et al., 2016b) in which less than a third of the laughs immediately follow their referents. Instead, the laugh can occur before, during or after the laughable with wide time ranges. In addition, laughter does "interrupt" speech: we frequently start laughing in the middle of an utterance of the interlocutor or of ourselves (often speech-laugh). Our results challenge the assumption that what laughter follows is what it is about, and thus question previous claims based on this assumption.
In future work, we will study to what extent laughter-laughable alignment differs by the function/effect of laughter, and what the limit is for the "free" alignment. This work may be useful for dialogue systems which allows a computer agent to generate laughter at appropriate times depending on the type and location of the laughable.