*Workshop Description*
All research exists within a larger context. Progress is made by “standing on the shoulders of giants” [Scotchmer, 1991] — building on the foundations laid by earlier researchers. In NLP it has been said that research is “not so much going round in circles as ascending a spiral” [Spark-Jones, 1994], while citation analysis of recent work suggests that research progress looks more like a series of intertwined staircases [Hearst 2018].
Though this context is critical to achieving and fully understanding progress in NLP, it can be increasingly difficult in the current research climate---with rapid publication rates and concise paper formats---to recognize the larger story to which a given paper or thread of research is connected. Papers typically include Related Work sections describing how they connect to prior work---but authors lack space and incentive to provide fully exhaustive coverage of related work, or to spend time carefully reconciling seemingly conflicting findings. Similarly, while researchers may be cognizant of how each of their own papers relates to their broader research agenda, it can be more difficult for readers of individual papers to appreciate the larger story told by a researcher's body of work. In omitting this larger context, we risk missing key insights provided by threads of work in aggregate. This leads us to misinterpret the significance of individual papers---and in the worst case, causes researchers unaware of prior work to reinvent the wheel or repeat past mistakes.
The goal of “The Big Picture” workshop is to provide a dedicated venue for exploring and distilling these broader NLP research narratives. We invite researchers to reflect on how their individual contributions fit within the overall research landscape, as well as what stories they are telling with their bodies of research — and to take this opportunity to communicate these research narratives to the broader community.
Additionally, in producing content focused on outlining higher-level narratives, the workshop aims to facilitate understanding of the trajectories and insights within the field of NLP, particularly for newcomers and outsiders to the field, in ways that individual research papers typically do not.
We welcome the following three types of papers for the workshop (but other topics that are similar in spirit are also welcome):
* Cross-lab collaborations – papers reconciling or distilling findings from different groups. Often different labs publish work on similar topics. However, there are almost always differences between such works, and familiarity with the precise differences and similarities is mainly limited to the original authors. In many cases, papers produce seemingly contradictory findings, or authors may register strong disagreements, which may be able to be clarified or reconciled through examination and discussion of the different approaches. Example topics that could benefit from such discussions include the question of how to measure bias in models and whether template-based datasets are sufficient [Nangia et al., 2020, Nadeem et al., 2021], which was debated in subsequent work [Blodgett et al., 2021]; how to measure social ethics and norms in models [Jiang et al., 2021] and whether we should even try doing so [Talat et al., 2022]; or whether language models can learn meaning solely from textual inputs [Bender and Koller 2020]. We encourage researchers to seek out and collaborate with researchers working in the same area, and to submit papers reporting the findings from these collaborations or discussions.
* Thesis-like papers – papers distilling a larger body of work from a given researcher. This category is particularly geared toward advanced PhD students (or recent graduates) who have a body of work on which their dissertation is, or will be, based. This workshop provides an opportunity for such researchers to distill their research contributions into a single coherent and concise story that conveys the bigger picture of their dissertation. Compared to existing venues such as doctoral consortiums or student research workshops, we encourage submitters not just to share a polished narrative of their work, but to highlight the limitations of early work, the surprising insights, and the lessons learned.
* Within-lab collaborations – papers distilling a larger body of work from a given research group. Principal investigators typically receive grants (e.g. ERC, NSF, etc.) to work on certain topics and applications, and have multiple students, collaborators, or employees who execute the larger research vision. This workshop provides an opportunity for such researchers and research groups to reflect on the work they have completed, to convey the larger story emerging from their body of work, and to let the community know what worked, what did not, and what comes next. Since these projects are not often discussed with the wider community, except in individual papers or in some invited talks, this venue will provide an opportunity to convey this larger narrative.
Feel free to reach out to the organizers at the email below if you are not sure whether a specific topic is well-suited for submission.
*Submission Instructions*
We are accepting a single type of submission:
Archival papers of up to 8 pages + references + appendix. These are papers reporting on completed, original, and unpublished research. Note that submitted papers should not be standard conference papers but should be associated with one of the categories described above.
Accepted papers are expected to be presented at the workshop and will be published in the workshop proceedings of the ACL Anthology, meaning they cannot be published elsewhere. They should report on obtained results rather than intended work.
Accepted submissions will be presented at the workshop as posters or as oral presentations (determined by the program committee).
Dual submissions are not allowed
*Important dates*
End of August 2023: Paper submission deadline
October 2023: Notification of acceptance
December 2023: Big Picture Workshop at EMNLP 2023
*Note*: These dates are estimated based on EMNLP 2022, and are likely to change once more details about EMNLP 2023 are announced.
*Website*
For the most recent information about the workshop check our website:
https://www.bigpictureworkshop.com/
*Contact*
Please contact the organizers at big-picture-workshop [at] googlegroups.com with any questions.