Stefan Riezler


2019

pdf bib
Self-Regulated Interactive Sequence-to-Sequence Learning
Julia Kreutzer | Stefan Riezler
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

Not all types of supervision signals are created equal: Different types of feedback have different costs and effects on learning. We show how self-regulation strategies that decide when to ask for which kind of feedback from a teacher (or from oneself) can be cast as a learning-to-learn problem leading to improved cost-aware sequence-to-sequence learning. In experiments on interactive neural machine translation, we find that the self-regulator discovers an đťś–-greedy strategy for the optimal cost-quality trade-off by mixing different feedback types including corrections, error markups, and self-supervision. Furthermore, we demonstrate its robustness under domain shift and identify it as a promising alternative to active learning.

pdf bib
Interactive-Predictive Neural Machine Translation through Reinforcement and Imitation
Tsz Kin Lam | Shigehiko Schamoni | Stefan Riezler
Proceedings of Machine Translation Summit XVII Volume 1: Research Track

pdf bib
Joey NMT: A Minimalist NMT Toolkit for Novices
Julia Kreutzer | Joost Bastings | Stefan Riezler
Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP): System Demonstrations

We present Joey NMT, a minimalist neural machine translation toolkit based on PyTorch that is specifically designed for novices. Joey NMT provides many popular NMT features in a small and simple code base, so that novices can easily and quickly learn to use it and adapt it to their needs. Despite its focus on simplicity, Joey NMT supports classic architectures (RNNs, transformers), fast beam search, weight tying, and more, and achieves performance comparable to more complex toolkits on standard benchmarks. We evaluate the accessibility of our toolkit in a user study where novices with general knowledge about Pytorch and NMT and experts work through a self-contained Joey NMT tutorial, showing that novices perform almost as well as experts in a subsequent code quiz. Joey NMT is available at https://github.com/joeynmt/joeynmt.

pdf bib
Learning Neural Sequence-to-Sequence Models from Weak Feedback with Bipolar Ramp Loss
Laura Jehl | Carolin Lawrence | Stefan Riezler
Transactions of the Association for Computational Linguistics, Volume 7

In many machine learning scenarios, supervision by gold labels is not available and conse quently neural models cannot be trained directly by maximum likelihood estimation. In a weak supervision scenario, metric-augmented objectives can be employed to assign feedback to model outputs, which can be used to extract a supervision signal for training. We present several objectives for two separate weakly supervised tasks, machine translation and semantic parsing. We show that objectives should actively discourage negative outputs in addition to promoting a surrogate gold structure. This notion of bipolarity is naturally present in ramp loss objectives, which we adapt to neural models. We show that bipolar ramp loss objectives outperform other non-bipolar ramp loss objectives and minimum risk training on both weakly supervised tasks, as well as on a supervised machine translation task. Additionally, we introduce a novel token-level ramp loss objective, which is able to outperform even the best sequence-level ramp loss on both weakly supervised tasks.

2018

pdf bib
Can Neural Machine Translation be Improved with User Feedback?
Julia Kreutzer | Shahram Khadivi | Evgeny Matusov | Stefan Riezler
Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 3 (Industry Papers)

We present the first real-world application of methods for improving neural machine translation (NMT) with human reinforcement, based on explicit and implicit user feedback collected on the eBay e-commerce platform. Previous work has been confined to simulation experiments, whereas in this paper we work with real logged feedback for offline bandit learning of NMT parameters. We conduct a thorough analysis of the available explicit user judgments—five-star ratings of translation quality—and show that they are not reliable enough to yield significant improvements in bandit learning. In contrast, we successfully utilize implicit task-based feedback collected in a cross-lingual search task to improve task-specific and machine translation quality metrics.

pdf bib
Document-Level Information as Side Constraints for Improved Neural Patent Translation
Laura Jehl | Stefan Riezler
Proceedings of the 13th Conference of the Association for Machine Translation in the Americas (Volume 1: Research Papers)

pdf bib
A Dataset and Reranking Method for Multimodal MT of User-Generated Image Captions
Shigehiko Schamoni | Julian Hitschler | Stefan Riezler
Proceedings of the 13th Conference of the Association for Machine Translation in the Americas (Volume 1: Research Papers)

pdf bib
Reliability and Learnability of Human Bandit Feedback for Sequence-to-Sequence Reinforcement Learning
Julia Kreutzer | Joshua Uyheng | Stefan Riezler
Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

We present a study on reinforcement learning (RL) from human bandit feedback for sequence-to-sequence learning, exemplified by the task of bandit neural machine translation (NMT). We investigate the reliability of human bandit feedback, and analyze the influence of reliability on the learnability of a reward estimator, and the effect of the quality of reward estimates on the overall RL task. Our analysis of cardinal (5-point ratings) and ordinal (pairwise preferences) feedback shows that their intra- and inter-annotator α-agreement is comparable. Best reliability is obtained for standardized cardinal feedback, and cardinal feedback is also easiest to learn and generalize from. Finally, improvements of over 1 BLEU can be obtained by integrating a regression-based reward estimator trained on cardinal feedback for 800 translations into RL for NMT. This shows that RL is possible even from small amounts of fairly reliable human feedback, pointing to a great potential for applications at larger scale.

pdf bib
Improving a Neural Semantic Parser by Counterfactual Learning from Human Bandit Feedback
Carolin Lawrence | Stefan Riezler
Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

Counterfactual learning from human bandit feedback describes a scenario where user feedback on the quality of outputs of a historic system is logged and used to improve a target system. We show how to apply this learning framework to neural semantic parsing. From a machine learning perspective, the key challenge lies in a proper reweighting of the estimator so as to avoid known degeneracies in counterfactual learning, while still being applicable to stochastic gradient optimization. To conduct experiments with human users, we devise an easy-to-use interface to collect human feedback on semantic parses. Our work is the first to show that semantic parsers can be improved significantly by counterfactual learning from logged human feedback data.

2017

pdf bib
Bandit Structured Prediction for Neural Sequence-to-Sequence Learning
Julia Kreutzer | Artem Sokolov | Stefan Riezler
Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

Bandit structured prediction describes a stochastic optimization framework where learning is performed from partial feedback. This feedback is received in the form of a task loss evaluation to a predicted output structure, without having access to gold standard structures. We advance this framework by lifting linear bandit learning to neural sequence-to-sequence learning problems using attention-based recurrent neural networks. Furthermore, we show how to incorporate control variates into our learning algorithms for variance reduction and improved generalization. We present an evaluation on a neural machine translation task that shows improvements of up to 5.89 BLEU points for domain adaptation from simulated bandit feedback.

pdf bib
A Shared Task on Bandit Learning for Machine Translation
Artem Sokolov | Julia Kreutzer | Kellen Sunderland | Pavel Danchenko | Witold Szymaniak | Hagen FĂĽrstenau | Stefan Riezler
Proceedings of the Second Conference on Machine Translation

pdf bib
Counterfactual Learning from Bandit Feedback under Deterministic Logging : A Case Study in Statistical Machine Translation
Carolin Lawrence | Artem Sokolov | Stefan Riezler
Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing

The goal of counterfactual learning for statistical machine translation (SMT) is to optimize a target SMT system from logged data that consist of user feedback to translations that were predicted by another, historic SMT system. A challenge arises by the fact that risk-averse commercial SMT systems deterministically log the most probable translation. The lack of sufficient exploration of the SMT output space seemingly contradicts the theoretical requirements for counterfactual learning. We show that counterfactual learning from deterministic bandit logs is possible nevertheless by smoothing out deterministic components in learning. This can be achieved by additive and multiplicative control variates that avoid degenerate behavior in empirical risk minimization. Our simulation experiments show improvements of up to 2 BLEU points by counterfactual learning from deterministic bandit feedback.

2016

pdf bib
Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning
Stefan Riezler | Yoav Goldberg
Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning

pdf bib
A Corpus and Semantic Parser for Multilingual Natural Language Querying of OpenStreetMap
Carolin Haas | Stefan Riezler
Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

pdf bib
Learning to translate from graded and negative relevance information
Laura Jehl | Stefan Riezler
Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers

We present an approach for learning to translate by exploiting cross-lingual link structure in multilingual document collections. We propose a new learning objective based on structured ramp loss, which learns from graded relevance, explicitly including negative relevance information. Our results on English German translation of Wikipedia entries show small, but significant, improvements of our method over an unadapted baseline, even when only a weak relevance signal is used. We also compare our method to monolingual language model adaptation and automatic pseudo-parallel data extraction and find small improvements even over these strong baselines.

pdf bib
NLmaps: A Natural Language Interface to Query OpenStreetMap
Carolin Lawrence | Stefan Riezler
Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: System Demonstrations

We present a Natural Language Interface (nlmaps.cl.uni-heidelberg.de) to query OpenStreetMap. Natural language questions about geographical facts are parsed into database queries that can be executed against the OpenStreetMap (OSM) database. After parsing the question, the system provides a text based answer as well as an interactive map with all points of interest and their relevant information marked. Additionally, we provide several options for users to give feedback after a question has been parsed.

pdf bib
A Post-editing Interface for Immediate Adaptation in Statistical Machine Translation
Patrick Simianer | Sariya Karimova | Stefan Riezler
Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: System Demonstrations

Adaptive machine translation (MT) systems are a promising approach for improving the effectiveness of computer-aided translation (CAT) environments. There is, however, virtually only theoretical work that examines how such a system could be implemented. We present an open source post-editing interface for adaptive statistical MT, which has in-depth monitoring capabilities and excellent expandability, and can facilitate practical studies. To this end, we designed text-based and graphical post-editing interfaces. The graphical interface offers means for displaying and editing a rich view of the MT output. Our translation systems may learn from post-edits using several weight, language model and novel translation model adaptation techniques, in part by exploiting the output of the graphical interface. In a user study we show that using the proposed interface and adaptation methods, reductions in technical effort and time can be achieved.

pdf bib
Learning Structured Predictors from Bandit Feedback for Interactive NLP
Artem Sokolov | Julia Kreutzer | Christopher Lo | Stefan Riezler
Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

pdf bib
Multimodal Pivots for Image Caption Translation
Julian Hitschler | Shigehiko Schamoni | Stefan Riezler
Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

2015

pdf bib
Bag-of-Words Forced Decoding for Cross-Lingual Information Retrieval
Felix Hieber | Stefan Riezler
Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

pdf bib
Response-based Learning for Machine Translation of Open-domain Database Queries
Carolin Haas | Stefan Riezler
Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

pdf bib
QUality Estimation from ScraTCH (QUETCH): Deep Learning for Word-level Translation Quality Estimation
Julia Kreutzer | Shigehiko Schamoni | Stefan Riezler
Proceedings of the Tenth Workshop on Statistical Machine Translation

pdf bib
Integrating a Large, Monolingual Corpus as Translation Memory into Statistical Machine Translation
Katharina Wäschle | Stefan Riezler
Proceedings of the 18th Annual Conference of the European Association for Machine Translation

pdf bib
A Coactive Learning View of Online Structured Prediction in Statistical Machine Translation
Artem Sokolov | Stefan Riezler | Shay B. Cohen
Proceedings of the Nineteenth Conference on Computational Natural Language Learning

2014

pdf bib
Last Words: On the Problem of Theoretical Terms in Empirical Computational Linguistics
Stefan Riezler
Computational Linguistics, Volume 40, Issue 1 - March 2014

pdf bib
Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics
Shuly Wintner | Sharon Goldwater | Stefan Riezler
Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics

pdf bib
Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, volume 2: Short Papers
Shuly Wintner | Stefan Riezler | Sharon Goldwater
Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, volume 2: Short Papers

pdf bib
Response-based Learning for Grounded Machine Translation
Stefan Riezler | Patrick Simianer | Carolin Haas
Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

pdf bib
Learning Translational and Knowledge-based Similarities from Relevance Rankings for Cross-Language Retrieval
Shigehiko Schamoni | Felix Hieber | Artem Sokolov | Stefan Riezler
Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)

2013

pdf bib
Task Alternation in Parallel Sentence Retrieval for Twitter Translation
Felix Hieber | Laura Jehl | Stefan Riezler
Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)

pdf bib
Multi-Task Learning for Improved Discriminative Training in SMT
Patrick Simianer | Stefan Riezler
Proceedings of the Eighth Workshop on Statistical Machine Translation

pdf bib
Boosting Cross-Language Retrieval by Learning Bilingual Phrase Associations from Relevance Rankings
Artem Sokokov | Laura Jehl | Felix Hieber | Stefan Riezler
Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing

2012

pdf bib
Joint Feature Selection in Distributed Stochastic Learning for Large-Scale Discriminative Training in SMT
Patrick Simianer | Stefan Riezler | Chris Dyer
Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

pdf bib
Structural and Topical Dimensions in Multi-Task Patent Translation
Katharina Waeschle | Stefan Riezler
Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics

pdf bib
Twitter Translation using Translation-Based Cross-Lingual Retrieval
Laura Jehl | Felix Hieber | Stefan Riezler
Proceedings of the Seventh Workshop on Statistical Machine Translation

2010

pdf bib
Query Rewriting Using Monolingual Statistical Machine Translation
Stefan Riezler | Yi Liu
Computational Linguistics, Volume 36, Issue 3 - September 2010

pdf bib
Learning Dense Models of Query Similarity from User Click Logs
Fabio De Bona | Stefan Riezler | Keith Hall | Massimiliano Ciaramita | Amaç Herdaǧdelen | Maria Holmqvist
Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics

2008

pdf bib
Wide-Coverage Deep Statistical Parsing Using Automatic Dependency Structure Annotation
Aoife Cahill | Michael Burke | Ruth O’Donovan | Stefan Riezler | Josef van Genabith | Andy Way
Computational Linguistics, Volume 34, Number 1, March 2008

pdf bib
Translating Queries into Snippets for Improved Query Expansion
Stefan Riezler | Yi Liu | Alexander Vasserman
Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008)

2007

pdf bib
Statistical Machine Translation for Query Expansion in Answer Retrieval
Stefan Riezler | Alexander Vasserman | Ioannis Tsochantaridis | Vibhu Mittal | Yi Liu
Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics

2006

pdf bib
Grammatical Machine Translation
Stefan Riezler | John T. Maxwell III
Proceedings of the Human Language Technology Conference of the NAACL, Main Conference

pdf bib
Book Reviews: New Developments in Parsing Technology, edited by Harry Bunt, John Carroll and Giorgio Satta
Stefan Riezler
Computational Linguistics, Volume 32, Number 3, September 2006

2005

pdf bib
On Some Pitfalls in Automatic Evaluation and Significance Testing for MT
Stefan Riezler | John T. Maxwell
Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization

2004

pdf bib
Speed and Accuracy in Shallow and Deep Stochastic Parsing
Ron Kaplan | Stefan Riezler | Tracy H. King | John T. Maxwell III | Alex Vasserman | Richard Crouch
Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics: HLT-NAACL 2004

pdf bib
Incremental Feature Selection and l1 Regularization for Relaxed Maximum-Entropy Modeling
Stefan Riezler | Alexander Vasserman
Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing

2003

pdf bib
Statistical Sentence Condensation using Ambiguity Packing and Stochastic Disambiguation Methods for Lexical-Functional Grammar
Stefan Riezler | Tracy H. King | Richard Crouch | Annie Zaenen
Proceedings of the 2003 Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics

pdf bib
The PARC 700 Dependency Bank
Tracy Holloway King | Richard Crouch | Stefan Riezler | Mary Dalrymple | Ronald M. Kaplan
Proceedings of 4th International Workshop on Linguistically Interpreted Corpora (LINC-03) at EACL 2003

2002

pdf bib
Parsing the Wall Street Journal using a Lexical-Functional Grammar and Discriminative Estimation Techniques
Stefan Riezler | Tracy H. King | Ronald M. Kaplan | Richard Crouch | John T. Maxwell III | Mark Johnson
Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics

2000

pdf bib
Using a Probabilistic Class-Based Lexicon for Lexical Ambiguity Resolution
Detlef Prescher | Stefan Riezler | Mats Rooth
COLING 2000 Volume 2: The 18th International Conference on Computational Linguistics

pdf bib
Lexicalized Stochastic Modeling of Constraint-Based Grammars using Log-Linear Measures and EM Training
Stefan Riezler | Detlef Prescher | Jonas Kuhn | Mark Johnson
Proceedings of the 38th Annual Meeting of the Association for Computational Linguistics

pdf bib
Exploiting auxiliary distributions in stochastic unification-based grammars
Mark Johnson | Stefan Riezler
1st Meeting of the North American Chapter of the Association for Computational Linguistics

1999

pdf bib
Inducing a Semantically Annotated Lexicon via EM-Based Clustering
Mats Rooth | Stefan Riezler | Detlef Prescher | Glenn Carroll | Franz Beil
Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics

pdf bib
Inside-Outside Estimation of a Lexicalized PCFG for German
Franz Beil | Glenn Carroll | Detlef Prescher | Stefan Riezler | Mats Rooth
Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics

pdf bib
Estimators for Stochastic “Unification-Based” Grammars
Mark Johnson | Stuart Geman | Stephen Canon | Zhiyi Chi | Stefan Riezler
Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics