Preparing SNACS for Subjects and Objects

Research on adpositions and possessives in multiple languages has led to a small inventory of general-purpose meaning classes that disambiguate tokens. Importantly, that work has argued for a principled separation of the semantic role in a scene from the function coded by morphosyntax. Here, we ask whether this approach can be generalized beyond adpositions and possessives to cover all scene participants—including subjects and objects—directly, without reference to a frame lexicon. We present new guidelines for English and the results of an interannotator agreement study.


Introduction
Studies of verbal argument structure have established some clear semantic correlations of syntactic relations like subject and object, and there are various approaches to expressing these generalizations using categorical semantic roles (Fillmore, 1968(Fillmore, , 1982Levin, 1993) or bundles of proto-properties (Dowty, 1991;Reisinger et al., 2015) that generalize across verbs. A parallel line of work ( §2) has looked at the meanings coded by grammatical phrase-markers such as prepositions and possessives and how to disambiguate them. These inquiries necessarily overlap because many prepositions mark verb arguments or modifiers. Consequently, insights from the study of prepositions/ case may improve the meaning representation of core syntactic arguments, or vice versa.
In this paper, we investigate whether SNACS (Schneider et al., 2018b), an approach to semantic disambiguation of adpositions and possessives, can be adapted to cover syntactically core grammatical relations (subjects and objects). We believe this may have several practical advantages for NLP.
First, many of the semantic labels in SNACS derive from VerbNet (Kipper et al., 2008) role labels.
However, VerbNet and other frame-semantic approaches like FrameNet (Fillmore and Baker, 2009) and PropBank (Palmer et al., 2005) assume a lexicon as a prerequisite for semantic role annotation. This can be an obstacle to comprehensive corpus annotation when out-of-vocabulary predicates are encountered. But is a lexicon really necessary for role annotation? A general-purpose set of role labels with detailed criteria for each can potentially bypass coverage limitations of lexicon-based approaches, while still supporting some degree of generalization across grammatical paraphrases.
Second, the nonreliance on a lexicon potentially simplifies the annotation process in some respects. For example, no explicit predicate disambiguation step is necessary, and the annotator does not need to consult frame-specific role definitions. 1 Third, the semantic criteria for SNACS labels are designed to be language-neutral, and investigations thus far suggest that they can be generalized to languages besides English (Hwang et al., 2017;Zhu et al., 2019). While this paper focuses on English, we see the future opportunity for crosslingual extension without the construction of new lexicons as a major advantage.
Finally, SNACS is unique in allowing two semantic labels per target, one reflecting a level of meaning closer to the grammatical coding, and the other at a deeper level associated with the predicate scene type ( §3). We show below that the SNACS analysis, while designed for PPs, can be extended to subjects and objects, to the extent that the coarsegrained inventory distinguishes roles in the scene.
We summarize SNACS in §2, and in §3 propose a strategy for adapting SNACS for English subjects and objects. This involves minor changes to the label inventory and new annotation guidelines for a variety of challenging phenomena. We conduct a pilot interannotator agreement study on Wikipedia articles ( §4) and release the annotations. 2

Background
The SNACS 3 hierarchy is a taxonomy of coarsegrained supersenses developed to mark semantic relations as expressed by adpositions (prepositions + postpositions) and possessives (Schneider et al., 2018b). The complete SNACS hierarchy is shown in figure 1 with our modifications highlighted.
SNACS includes the usual thematic relations (e.g., AGENT, THEME, RECIPIENT) and adjunct relations (e.g., TIME, LOCUS including locations, PURPOSE) used by most resources designed for SRL annotation. SNACS diverges from the general predicate-argument labeling standards in its inclusion of non-standard roles such as ORIGINATOR in creation (creator), transfer (giver) and communication (speaker) events, and labels regarding properties involved in a static relationship to one another (e.g., POSSESSION in "owner of the car").
Unlike labels used by efforts such as PropBank and FrameNet, SNACS labels are highly coarsegrained and generalize across various scenes and situations. This approach also differs from framealternation-based lexicons like VerbNet, which defines classes of verbs whose members exhibit similar syntactic alternations involving the same subset 2 https://github.com/adishalev/SNACS _ DMR _ IAA 3 Semantic Network of Adposition and Case Supersenses of roles. Instead, SNACS places the burden of semantics directly on a fixed set of supersenses, forgoing the use of frame (or class) definitions. The supersenses can be thought of as disambiguating coarse-grained adposition senses. The supersense labels effectively encapsulate-at a highly abstract/ schematic level-various basic scenarios that are important to language and grammar, such as transitive action, motion, unidirectional transfer/communication, and psychological events, as well as stative relations like possession, quantity, comparison, and identity. SNACS does not formalize a semantic core/non-core or argument/adjunct distinction, though roles in the PARTICIPANT hierarchy are typically core and roles in the CIRCUMSTANCE hierarchy are typically non-core in predicate-argument annotation schemes like PropBank and FrameNet.

Applying SNACS
We adopt the SNACS labels originally developed for disambiguating adpositions and possessives as exemplified in (1) and extend their use to annotate the subject and object of a verb as seen in (2). (1) a. The bagel was eaten by AGENT Jane.
b. Jane dined on THEME a bagel.
Following the construal approach, which is illustrated in table 1 for adpositions, we separate two semantic dimensions of an annotation target: Scene Role: What semantic role is most closely associated with the type of scene (typically indicated by the verb/predicate)? Function: What semantic role is most salient in the morphosyntactic coding of the phrase (with a grammatical relation like subject or object, or overt marking with closed-class morphology like adpositions and case)? Consider the following examples. Construal is notated by SCENE ROLE↝FUNCTION.
The scene role indicates the participation role of the target in the scene described by the verb. Jane is the RECIPIENT in a transfer scene in (3), and she is in a certain social relationship with Bingley (i.e., SOCIALREL) given the marriage scene in (4). The function label, on the other hand, captures the  Table 1: SNACS for adpositions/possessives (Schneider et al., 2018b,a). The scene role and function annotations are labels from figure 1 and are often but not always congruent for a particular token. The function annotation reflects the semantics of the morphosyntactic coding (such as the choice of adposition). Note that, especially for adnominal PPs and genitives, the governor sometimes does not lexically denote an event or state; rather, a semantic relation such as possession or part-whole is indicated by the morphosyntax.
orthogonal dimsension of agency which is more closely tied to syntactic realization: Jane is the AGENT of the buying action, while the book is the THEME in (3); Jane and Bingley are the THEMEs of the marriage in (4). Further examples are shown in figure 2. In many cases, the scene role will be identical to the function. These are called congruent construals. But in other cases, they can differ, as illustrated in table 1.
In the rest of the section, we discuss a few difficult cases while assessing SNACS labels for the annotation of subject and objects, and decisions made regarding these challenges including slight deviations from the latest SNACS standards. Scene role prioritization. In some cases, multiple supersenses are equally applicable at the scene level. In such cases, we give highest priority to more complex and less frequent scene types such as transfer (ORIGINATOR, RECIPIENT) or employeeorganization (or social) relations (ORGMEMBER, ORG, SOCIALREL). The causal roles (AGENT, INSTRUMENT, THEME), if appearing in the scene position, are prioritized next. The highly frequent locative scenes (LOCUS, SOURCE, GOAL) are given the lowest priority. In example (10), the subject "I" could be considered either a metaphorical source location of the recommendation (i.e., SOURCE) or can be considered the speaker in a communication event (i.e., ORIGINATOR). The latter scene is prioritized, and the scene roles ORIGI-NATOR, TOPIC (i.e., the message), and RECIPIENT reflect the prioritized choice. In (11,12), Jane is arguably a GOAL of motion by virtue of being a RECIPIENT of something physical. We do not use GOAL as the scene role, however, if RECIPIENT applies. In (13), Mary can be understood as part of the cast (which is an organization) or as located within the cast. We prioritize the former. Other pairs that tend to overlap include: RECIPIENT/BENEFICIARY, closely correlated when someone is given or told something for their benefit or harm-we prioritize RECIPIENT for the scene role; and STIMULUS/TOPIC, closely correlated when a thought or message triggers an emotional reaction-we prioritize STIMULUS for the scene role. If two equally prioritized scenes are in conflict with one another, we rely on the semantics of the predicate to disambiguate the scene. Note that in (14), CJ is likely an employee of the White House. However, CJ is not considered the ORGMEMBER as the verb "brief" does not intrinsically conventionalize the employee-organization relationship in its semantics. With a predicate that conventionally encodes employment or some other stable relationship-employ, hire, work for/at, etc.-ORGMEMBER would be annotated.  labels such as AGENT or THEME to appear multiple times in a given scene, opting for the use of a "Co-" label for the second participant sharing the same role (e.g., CO-AGENT). In applying SNACS guidelines for subjects and objects, this became untenable, as "Co-" prefixation could apply to a good majority of the PARTICIPANT labels, threatening a quick proliferation of the supersenses. E.g., (4) would require CO-SOCIALREL, (16) would require CO-EXPERIENCER, and so forth. In an effort to keep the supersense inventory limited, we diverge from the latest SNACS standards to allow role duplication in a scene. This is allowed even when targets assigned the same role are not fully symmetric or are qualitatively distinct as in (17) Thematic hierarchy. As discussed above, the function label generally reflects AGENT-THEME relations of a proposition. More specifically, we annotate all subjects and direct objects with a function in the following thematic hierarchy: {AGENT, CAUSER} > {INSTRUMENT, MEANS} > {THEME, TOPIC, COST}. In a transitive clause, the supersense of the subject cannot be ranked lower than the direct object (e.g., a subject construed as a THEME cannot have a direct object construed as an AGENT). Indirect objects in the English double object construction 4 are treated as RECIPIENT construals. Copular sentences. These are treated differently from non-copular sentences. The English copula relates a subject to an object in what is semantically an identificational (22a) or predicational (22b) relationship. To these cases we assign IDENTITY-IDENTITY or GESTALT-CHARACTERISTIC at the scene level, respectively. Roughly speaking, IDEN-TITY indicates the identified or identifying category or referent, and CHARACTERISTIC indicates a property being ascribed to the GESTALT: Open issues. The unresolved problem of causatives and caused-motion constructions is discussed in appendix A.

Interannotator Agreement Study
Data. We piloted our guidelines using a sample of 100 scenes from the English UCCA-annotated Wiki corpus 5 as detailed by Abend and Rappoport (2013). UCCA is a scheme for annotating coarsegrained predicate-argument structure such that syntactically varied paraphrases and translations should receive similar analyses. It captures both static and dynamic scenes and their participants, but does not mark semantic roles. Annotators. Four annotators (A, B, C, D), all authors of this paper, took part in this study. All are computational linguistics researchers.
Datasets. Prior to development of guidelines for subjects and objects, one of the annotators (Annotator A) sampled 106 Wiki documents (44k tokens) and tagged all 10k instances of UCCA Participants 6 with a supersense based on the existing guidelines 5 http://cs.huji.ac.il/~oabend/ucca.html; the Wikipedia corpus contains 369 documents (biographies of entertainment industry figures) with 159k tokens and 36k Participant units. 6 The UCCA category Participant is broader than the PAR-TICIPANT supersense, also including locations, for example.  for adpositions. This preliminary dataset was used to stimulate discussion for developing guidelines for subjects and objects. Once the new guidelines were written, four annotators first annotated a practice dataset of 48 UCCA Participant tokens, then adjudicated disagreements through discussion and clarified certain policies before annotating a final sample of 100 tokens for measuring agreement. 7 Participant units were sampled based on the preliminary annotation, stratified across subtypes of PARTICIPANT and CONFIGURATION to ensure diversity. In the final sample, the syntactic distribution is as follows: 31 subjects (including 4 passive subjects and 6 copular subjects); 26 objects (including 1 indirect object and 2 copular complements); 42 PPs; and 1 possessive.
Coverage of the hierarchy. Under the PARTIC-IPANT tree in the hierarchy, there are 12 supersenses, of which 11 were used as scene roles and 9 as functions. (By design, PARTICIPANT itself is never used and exists only to organize the hierarchy.) The CONFIGURATION tree includes 19 supersenses, of which 14 were used as scene roles and 10 as functions. In the CIRCUMSTANCE tree-which primarily applies to syntactic adverbials-GOAL, LOCUS, SOURCE, MANNER, MEANS, and CIR-CUMSTANCE were all used as functions, and all but SOURCE also appeared as a scene role. Quantitative IAA results. We first compare agreement on two subsamples: the subject/object Participants, and the prepositional phrase Participants. Pairwise Cohen's κ scores appear in table 2. Subjects/objects: For the scene role, all annotators agree on 46% of items (26/57), and at least 3 annotators on 84%. For the function, 51% have total agreement, and 86% have a majority. Average pairwise κ is 0.66 for scene and 0.61 for function. PPs: At the scene level, 48% (20/42) have total agreement, and 71% have a majority. For the function, 64% have total agreement, and 88% have a majority. Average pairwise κ is 0.64 for scene and 0.77 for function. 7 4 tokens where annotators noticed a problem with the UCCA annotation were discarded and replaced.
Thus subjects/objects (SOs) receive higher scene role agreement than PPs-somewhat surprising given that the labels were originally designed for prepositions! This may be an artifact of the particular sample, or may indicate that the scene role is more intuitive for SOs than for PPs. PPs have higher agreement than SOs with respect to function; this may be due to some difficulty deciding between AGENT and THEME for the function of SOs, plus the availability of extensive guidelines/examples for prepositional SNACS annotation. 8 Disagreements involving agentivity. We found it can be difficult to choose between AGENT and THEME for the function of a subject with borderline agentivity, e.g., in scenes of befriending someone or forming a musical group with others. Likewise, the line between AGENT and THEME for the function can be unclear in cognition/perception scenes like [She] enjoyed the fame and [She] saw the social scene as tedious and superficial. We decided the annotator should consider whether the scene involves judgment or is more of a passive experience; EXPERIENCER↝THEME would thus apply to the first example and EXPERIENCER↝AGENT to second. 9 Finally, the line between CAUSER and IN-STRUMENT can be unclear in sentences like I was hit [by a car] and I was quoted [by a magazine]. UCCA issues. We found a handful of UCCA annotation errors-primarily where two verbs were analyzed as separate scenes but the first ought to be considered a light verb. A more interesting case was the relation between the two bolded expressions in William S. Paley set terms that included. . . ownership of the negative at the end of the contract. The UCCA annotation treats William S. Paley as a Participant of ownership (i.e., the owner). Though POSSESSOR is a natural scene role for the owner of something, we concluded that this was an indirect inference not suitable for annotating with a function.

Conclusion
We explored whether a system for semantic relation annotation can be extended beyond prepositions and possessives to cover English subjects and objects. While initial annotation results are promising, further work is needed to substantiate the approach on a larger scale, and ideally in multiple languages.

A Open Issues
Sometimes a sentence will construe a scene as involving more arguments than a predicate normally licenses, as in the following causative or causedmotion examples (Goldberg, 2006) So far, we have posited the scene to be the situation or event described by the predicate. The problem is that in addition to the scene evoked by the verb (sneezing in (23) and jumping in (24)), there is an added caused-motion scene whose semantics derives from the construction. Should there be an indication that the sneezer is also the causer of motion, and that the ultimate causer of jumping is separate from the impelled jumper? One possible solution would be to add implicit predicates so the verb-and construction-triggered scenes would be annotated separately. A different solution may be to relax the definition of what constitutes a scene to allow for non-predicate-driven scenes as well.