Ambiguity in Explicit Discourse Connectives

Discourse connectives are known to be subject to both usage and sense ambiguity, as has already been discussed in the literature. But discourse connectives are no different from other linguistic expressions in being subject to other types of ambiguity as well. Four are illustrated and discussed here.


Introduction
Discourse connectives, like other linguistic expressions, are subject to ambiguity. Two types of ambiguity usage ambiguity, whether or not a given token is serving as a discourse connective in its context, and sense ambiguity, what discourse relation(s) a given token is signalling -were the subject of a study by Pitler and Nenkova (2009), who showed how syntactic features could help resolve them both.
But discourse connectives are no different from other linguistic expressions in being subject to other types of ambiguity as well. Four of them are discussed here, as a way of encouraging researchers to determine whether existing disambiguation methods suffice to handle them or whether the methods need to be extended. Ignoring the full range of ambiguity of discourse connectives can lead to discourse relations being mis-labelled both manually (during annotation) and automatically (during discourse parsing).
As background to presenting these ambiguities, Section 2 briefly reviews the original Penn Discourse TreeBank (the PDTB-2), the findings of Pitler and Nenkova (2009), and how the recently released PDTB-3 extends and, in some cases corrects, annotation in the PDTB-2. We then turn to four additional types of discourse connective ambiguity that have been discussed in the context of other linguistic forms. Section 3 discusses part-of-speech ambiguity, which can affect how a given token functions as a discourse connective. Section 4 discusses multi-word ambiguity, where a sequence of tokens can be ambiguous between a sequence of separate elements and a single multi-word discourse connective. Section 5 discusses a scope ambiguity that affects the sense of discourse connectives. Finally, Section 6 discusses semantic role ambiguity involving the arguments of certain CONCESSION relations.

PDTB-2
The Penn Discourse Treebank (Prasad et al., 2008) was created as the largest public repository of annotated discourse relations (over 43K), including over 18.4K signalled by explicit discourse connectives (coordinating or subordinating conjunctions, or discourse adverbials). All relations in the corpus are labelled with either one or two senses from a three-level sense hierarchy, whose top level comprised four non-terminal senses: EXPANSION, COMPARISON, CONTINGENCY and TEMPORAL. Most discourse relations were labelled with terminal senses, except where annotators were unable to decide and backed off to a level-2 (or in some cases, a top-level) sense. Discourse relations consisted of two arguments labelled Arg1 and Arg2, with each relation anchored by either an explicit discourse connective or adjacency. In the latter case, annotators inserted one or more implicit connectives that signalled the sense(s) they inferred to hold between the arguments. The approach in the PDTB-2 is agnostic about any higher-level discourse structure, and as such, made no attempt to build a tree or graph structure of relations over the text as a whole. The size and availability of the PDTB-2 spawned the field of shallow discourse parsing, as in the 2015 and 2016 CoNLL shared tasks (Xue et al., 2015(Xue et al., , 2016, as well as the development of similar resources for other languages, including Chinese, Hindi, and Turkish. An in-depth discussion of the PDTB-2 can be found in (Prasad et al., 2014). Pitler and Nenkova (2009) showed how syntactic features could be used in disambiguating both usage ambiguity and sense ambiguity. To understand these types of ambiguity, consider the word since. Ex. 1 illustrates its non-discourse usage, where since is simply a temporal preposition. Both Ex. 2 and Ex. 3 illustrate discourse usages and also the sense ambiguity of since, signalling a purely temporal relation in Ex. 2 and a purely causal relation in Ex. 3.
(2) There have been over 100 mergers since the most recent wave of friendly takeovers ended.
(3) It was a far safer deal since the company has a healthier cash flow.
Using data in the PDTB-2, Pitler and Nenkova (2009) showed that usage ambiguity can be resolved with high accuracy, as can sense ambiguity with respect to the four top-level sense classes (cf. Section 2.1). (N.B. They took multi-labelled tokens to be classified correctly if at least one of the senses was correctly identified.) They showed how high accuracy could be achieved in both disambiguation tasks by using both the token itself and its syntactic features in classification. Features included the syntactic category of the node dominating all and only the token itself, the category of its immediate parent, and the categories of its siblings. When they added interactions between connectives and syntactic features, and interactions between the features themselves, accuracy increased over 10 points and f-score, nearly 20 points.
Since Pitler & Nenkova's results are not incompatible with other types of discourse connective ambiguity, their work is a good jumping off point for experimenting with the additional types of discourse connective ambiguity we discuss here.

PDTB-3
The PDTB-3 1 contains ∼12.5K more intra-sentential relations (i.e., ones that lie wholly within the projection of a top-level S-node) and ∼1K more inter-sentential relations than the PDTB-2 (Webber et al., 2019). New senses have been added to the sense hierarchy (Table 1) and used for annotating new tokens, as well as for re-annotating existing tokens.
Newly annotated intra-sentential relations include ones between the conjuncts of conjoined verb phrases and conjoined clauses; ones between free or headed adjuncts and the clauses they adjoin to; ones associated with subordinators such as in order, prepositions such as with, for, and in; and ones between infinitival clauses (or other subordinating structures) and their matrix clause. New annotation also includes explicitly marked question-response pairs, and lexico-syntactic constructions that are unambiguous signals of particular discourse relations, such as the so-construction, signalling RESULT (Ex. 4), the too-construction, signalling NEGATIVE-RESULT (Ex. 5), and auxiliary inversion, signalling a CONDITIONAL relation (Ex. 6).  Differences in how discourse relations are annotated in the PDTB-2 and the PDTB-3 reflect (1) changes and/or additions to the sense hierarchy; (2) different criteria for choosing one sense label over another; and (3) rigorous attention to semantic consistency (Hollenstein et al., 2016), checking that similar tokens have been annotated in a similar way, thereby reducing annotation noise and improving what can be induced from the corpus.
Note that additions to the sense hierarchy have introduced new sense ambiguities that weren't present in the PDTB-2. A case in point is the discourse adverbial as well, all of whose tokens were senseannotated EXPANSION.CONJUNCTION in the PDTB-2 (cf. Ex. 7 and Ex. 8). With the new sense COM-PARISON.SIMILARITY, as well is now ambiguous between conveying EXPANSION.CONJUNCTION, which Ex. 7 is still taken to do, and COMPARISON.SIMILARITY, which Ex. 8 was re-annotated as.
(7) There is speculation that property/casualty firms will sell even more munis as they scramble to raise cash to pay claims related to Hurricane Hugo and the Northern California earthquake. Fundamental factors are at work as well. [wsj 0671] (8) "They continue to pay their bills and will do so," says Ms. Sanger. "We're confident we'll be paying our bills for spring merchandise as well." [wsj 1002] The PDTB-3 records the provenance of each token. This shows that, of the ∼53600 tokens annotated in the PDTB-3, ∼57% are unchanged from the PDTB-2, ∼19% (∼9900 tokens) have been changed in some way from their earlier annotation, and the remaining 24% are new to the PDTB-3. Provenance allows us to compare the use of new senses in annotating new tokens and in re-annotating existing tokens. For example, the new sense COMPARISON.SIMILARITY was used in annotating 135 new tokens and in re-annotating 68 tokens, of which 41 were associated with explicit connectives. Of the 41, 18 involved the discourse adverbial similarly; 10, the subordinating conjunction as if ; while the rest involved tokens of as, as though, as well, much as, just as meanwhile and while. While similarly thus unambiguously signals COMPARISON.SIMILARITY, for these other connectives, the new sense has meant a new ambiguity. These additional sense ambiguities argue for re-applying Pitler & Nenkova's analysis to the PDTB-3.
Part-of-Speech (PoS) affects how three items function as discourse connectives: since, before, and however. Since is ambiguous between a subordinating conjunction (PoS=IN), as in Ex. 9, and an adverbial (PoS=RB), as in Ex. 10.
(9) However, since Eastern first filed for Chapter 11 protection March 9, it has consistently promised to pay creditors 100 cents on the dollar. [wsj 0475] (10) His company, Misa Manufacturing Inc., was forced to seek protection from creditors under federal bankruptcy law in 1987 and has since been liquidated. [wsj 1830] This ambiguity also affects the sense of since. As a subordinating conjunction, since signals either REASON or (temporal) SUCCESSION (cf. Ex 9), while as a discourse adverbial, its temporal sense is the reverse -PRECEDENCE (cf. Ex. 10).
The same holds for before, which is also ambiguous between a subordinating conjunction (PoS=IN) and an adverbial (PoS=RB). As a subordinating conjunction, it conveys PRECEDENCE (cf. Ex. 11), while as a discourse adverbial, it conveys the reverse -SUCCESSION (cf. Ex. 12).
(11) They said they wanted to wait for the outcome of any government investigation before deciding what to do. Finally, however is ambiguous between a simple adverbial (PoS=RB) and a WH-Adverb subordinator (Pos=WRB). The latter is shown in Ex. 13.
(13) The 1987 crash was "a false alarm however you view it," says University of Chicago economist Victor Zarnowitz. [wsj 2397] As a simple discourse adverbial, the most common sense of however is COMPARISON.CONCESSION.ARG2-AS-DENIER. As a subordinator, the most common sense of however is the reverse, COMPARISON.CONCESSION.ARG1-AS-DENIER. 3 As for resolving these ambiguities, reliable disambiguation of their usage as discourse connectives only requires correct PoS-tagging to disambiguate how they are functioning as discourse connectives.

Multi-word Expression Ambiguity
Another ambiguity arises when a multi-word sequence can be analyzed either as a sequence of separate elements or as a single multi-word connective. Four sequences (but then, only to, or otherwise and but also) are ambiguous in this way.
Take but then: It can be interpreted as as a sequence of connectives, with but conveying COM-PARISON.CONTRAST or COMPARISON.CONCESSION.ARG2-AS-DENIER and then conveying TEMPO-RAL.ASYNCHRONOUS.PRECEDENCE, as in (14) Small businesses say a recent trend is like a dream come true: more-affordable rates for employee-health insurance, initially at least. But then they wake up to a nightmare. [wsj 0518] Alternatively, it can be interpreted as a single multi-word connective that expresses COMPARISON.CONCESSION.ARG2-AS-DENIER, as in (15) To many, it was a ceremony more befitting a king than a rural judge seated in the isolated foothills of the southern Allegheny Mountains. But then Judge O'Kicki often behaved like a man who would be kingand, some say, an arrogant and abusive one. [wsj 0267] (N.B. The CONCESSION label corresponds to a paraphrase with even though -e.g. "Even though it was a ceremony more befitting a king than a rural judge seated in the isolated foothills of the southern Allegheny Mountains, Judge O'Kicki often behaved like a man who would be king . . . ". Multi-word but then also implies that "you shouldn't be surprised at this because", but this is not something that was annotated in the PDTB-3.) Another ambiguous sequence is only to. On the one hand, only can be interpreted as modifying to, as it does in only because, only when, etc., indicating that Arg2 is the only thing in the given relation with Arg1, cf.
(16) Tax-exempt airport and street-corner solicitations were intended only to provide start-up funds. [wsj 0282] On the other hand, only to can be interpreted as a single multi-word connective conveying that Arg2 is a surprising, unexpected situation that follows Arg1, as in (17) Two former secretaries told the grand jury they were summoned to the judge's chambers on separate occasions to take dictation, only to find the judge in his bikini underwear. [wsj 0267] This is indicated by only to being labelled both PRECEDENCE and CONCESSION.ARG2-AS-DENIER.
A third ambiguous sequence is or otherwise. It can be analyzed as two separate connectives, with otherwise expressing CONTINGENCY.NEGATIVE-CONDITION.ARG1-AS-NEGCOND, as in Example 18 (paraphrasable as "if you don't stay in the center of the path, you might trip and fall."), or as a single multi-word connective, with or otherwise conveying that the disjuncts are mutually exclusive and that their union covers the full set, as in Example 19.  The final ambiguous sequence that we note here, but also (or in some cases, just the word but), sometimes appears as part of the paired connective not only . . . but also (cf. Ex. 20), usually senseannotated as CONJUNCTION.
(20) Market participants say investors are not only licking their wounds following the turbulence last week, but they have also been made nervous by two events in West Germany. [wsj 1187] Alternatively, the sequence can be analyzed as two distinct connectives -but, signalling CONTRAST or CONCESSION.ARG2-AS-DENIER, and also, signalling CONJUNCTION. This is how they have been labelled in wsj 0044 (Ex. 21).
(21) a. Editorials in the Greenville newspaper allowed that Mrs. Yeargin was wrong, but also said the case showed how testing was being overused. [wsj 0044] b. Editorials in the Greenville newspaper allowed that Mrs. Yeargin was wrong, but also said the case showed how testing was being overused. [wsj 0044] As with both usage and sense ambiguity, it would be useful to determine whether syntactic features might help distinguish whether a particular multi-word span should be analyzed as a single connective or separate elements.

Scope Ambiguity
Scope was only an issue in the PDTB-2 with respect to attribution, where a verb of attribution such as say or think might be superficially negated, while having the negation actually work to reverse the polarity of the attributed argument or relation (The PDTB Research Group, 2008). But scope is also a source of ambiguity in the PDTB-3, where it can affect the sense of to-clause constructions.
Absent modality or negation, a to-clause construction has a simple sense ambiguity. The to-clause can be Arg2 of either a CONTINGENCY.PURPOSE.ARG2-AS-GOAL relation (Ex. 22), or of a CONTIN-GENCY.CAUSE.RESULT relation (Ex. 23).
(22) The Galileo project started in 1977, and a number of project veterans were on hand to watch the launch.
[wsj 1817] (23) Georgia Gulf stock rose $1.75 a share yesterday to close at $51.25 a share. [wsj 0080] (By definition, PURPOSE requires a volitional agent, and generally can be paraphrased by inserting in order, while with RESULT, inserting therefore leads to a more appropriate paraphrase.) However, in the context of a modal (e.g., need, have to, must, require. etc.) or future tense (or present tense used as future), an additional ambiguity appears, whose disambiguation depends on whether the scope of the modal or future tense includes just Arg1 or both arguments. Specifically, if the scope includes just Arg1, annotators have taken the sense as being conditional (ARG2-AS-COND), because while the situation specified in the to-clause (Arg2) might be the agent's purpose, there is no assertion that it is so. This can be seen in the use of if as an appropriate paraphrase, as in Ex. 24-25 (24) Banks need a competitive edge to sell their products. [wsj 0238] paraphrase: Banks need a competitive edge if they are to sell their products.
(25) He said the index would have to be in the low 40% range for several months to be considered a forecast of recession. [wsj 0036] paraphrase: He said the index would have to be in the low 40% range for several months if it is to be considered a forecast of recession.
In contrast, if the modal or future operator seems best interpreted as scoping both arguments, as in (26) The two companies have been discussing a transaction under which Fresenius would buy Delmed stock for cash to bring its beneficial ownership to between 70% and 80% of Delmed's fully diluted common stock. [wsj 1066] paraphrase: . . . under which it would be the case that Fresenius buys Delmed stock for cash to bring its beneficial ownership to . . . then we are back to the original sense ambiguity between PURPOSE.ARG2-AS-GOAL and RESULT.
This same CONDITIONAL sense ambiguity also arises when there is negation or a question in Arg1, because its scope is again ambiguous between being just over Arg1 or over both arguments. This can be manually disambiguated by seeing whether if can be appropriately inserted in a positive paraphrase of Arg1 (in the case of negation) or a non-question paraphrase of Arg1 (in a question context). If so, scope only extends over Arg1, and the sense is CONDITION.ARG2-AS-COND, as in (27) . . . which, unlike utilities, aren't regulated and therefore don't need government approval to construct new plants. [wsj 0560] paraphrase: . . . and therefore they need government approval if they are to construct new plants In the case of the question in Ex. 28, (28) Do you really need this much money to put up these investments? [wsj 0629] "You really need this much money if you are to put up these investments" was not considered an appropriate non-question paraphrase of the original: The question was taken to scope both arguments. As such, one is back to the original sense ambiguity of to-clause constructions between PURPOSE.ARG2-AS-GOAL and RESULT. Since here, an appropriate paraphrase involves in order -"You really need this much money in order to put up these investments" -PURPOSE.ARG2-AS-GOAL was taken to be an appropriate sense label. In all, of over 1600 relations whose Arg2 was a to-clause construction, about 9% were sense-labelled CONDITION.ARG2-AS-COND.
While scope cannot be disambiguated by purely syntactic means, disambiguating these cases may require methods that go beyond the purely syntax-based approach of Pitler and Nenkova (2009).

Semantic Role Ambiguity
In English, semantic role ambiguity has mainly been discussed in the context of "garden path" sentences (Konstas et al., 2014), where in there is an ambiguity as to whether the horse (the man) is in the agent role of main verb raced (served) or the patient role of raced (served) as head of a reduced relative clause. Where listeners consistently make the wrong choice, it is considered a "garden path" sentence.
A few verbs in English such as shame demonstrate semantic role ambiguity even without considering reduced relative clauses. As shown in Ex. 30, even after processing the direct object of shame (i.e., me), there is still an ambiguity as to who plays the role of shamer and who, the shamee. With discourse connectives, four subordinating conjunctions that can signal a CONCESSION relation although, though, even though, and while -show a similar ambiguity when they head a postposed subordinate clause. The ambiguity here is which clause raises the causal inference and which denies it. It is an ambiguity that does not appear with preposed subordinate clauses.
More specifically, we noted in Section 2.3 that some senses are asymmetric, meaning that the relation is directional. To capture this directionality, each asymmetric relation has two Level-3 senses, in one of which Arg1 plays the specified role, while in the other case, Arg2 does so (cf. Table 1).
In general, an explicit connective that signals an asymmetric sense does so unambiguously. For example, when otherwise signals exception, the exception is Arg1 (EXPANSION.EXCEPTION.ARG1-AS-EXCPT), as in Ex. 31. In contrast, when except signals exception, the exception is Arg2 (EXPANSION.EXCEPTION.ARG2-AS-EXCPT), as in Ex. 32.
(31) Twenty-five years ago the poet Richard Wilbur modernized this 17th-century comedy merely by avoiding "the zounds sort of thing," as he wrote in his introduction. Otherwise, the scene remained Celimene's house in 1666. [wsj 0936] (32) Boston Co. officials declined to comment on Moody's action on the unit's financial performance this year except to deny a published report that outside accountants had discovered evidence of significant accounting errors in the first three quarters' results.
While CONCESSION relations are asymmetric, the subordinating conjunctions although, though, even though and while are not always unambiguous signals. That is, when they head a preposed subordinate clause (402 tokens in the PDTB-3), they were taken as unambiguously signalling the relation COMPARISON.CONCESSION.ARG1-AS-DENIER, where the matrix clause (Arg1) denies the causal inference raised by the subordinate clause, Arg2, as in Ex. 33.
(33) The documents also said that although the 64-year-old Mr. Cray has been working on the project for more than six years, the Cray-3 machine is at least another year away from a fully operational prototype.
[wsj 0018] However, when postposed with respect to its matrix clause, there is an ambiguity as to whether the matrix clause (Arg1) plays the role of denying the causal inference raised in Arg2, as in Ex. 34, or whether the subordinate clause (Arg2) plays the role of denying the causal inference raised in Arg1, as in Ex. 35. In the PDTB-3, there are 324 tokens of postposed CONCESSION relations with one of these four connectives. Of these, 260 have been labelled CONCESSION.ARG1-AS-DENIER, as with comparable preposed subordinate clauses, while the remaining 64 have been labelled CONCESSION.ARG2-AS-DENIER. The only differences between the four connectives is their relative frequency with which they appear in post-position and the degree of ambiguity when they do. While further analysis should identify features that will help disambiguate the sense of post-posed CON-CESSIVES, it is nevertheless worth establishing that semantic role ambiguity is not limited to verbs.

Conclusion
We hope to have shown that discourse connectives are no different from other linguistic expressions in being subject to many types of ambiguity. Besides usage ambiguity and sense ambiguity (Pitler and Nenkova, 2009), we hope to have shown that discourse connectives are subject to ambiguities associated with parts-of-speech, multi-word expressions, scope and semantic roles. We hope this will now encourage researchers to explore whether existing disambiguation methods suffice to handle this larger range of discourse connective ambiguities or whether such methods need to be extended.