Tomasz Stanisławek

Also published as: Tomasz Stanislawek


2019

pdf bib
Named Entity Recognition - Is There a Glass Ceiling?
Tomasz Stanislawek | Anna Wróblewska | Alicja Wójcicka | Daniel Ziembicki | Przemyslaw Biecek
Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL)

Recent developments in Named Entity Recognition (NER) have resulted in better and better models. However, is there a glass ceiling? Do we know which types of errors are still hard or even impossible to correct? In this paper, we present a detailed analysis of the types of errors in state-of-the-art machine learning (ML) methods. Our study illustrates weak and strong points of the Stanford, CMU, FLAIR, ELMO and BERT models, as well as their shared limitations. We also introduce new techniques for improving annotation, training process, and for checking model quality and stability.

pdf bib
GEval: Tool for Debugging NLP Datasets and Models
Filip Graliński | Anna Wróblewska | Tomasz Stanisławek | Kamil Grabowski | Tomasz Górecki
Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP

This paper presents a simple but general and effective method to debug the output of machine learning (ML) supervised models, including neural networks. The algorithm looks for features that lower the evaluation metric in such a way that it cannot be ascribed to chance (as measured by their p-values). Using this method – implemented as MLEval tool – you can find: (1) anomalies in test sets, (2) issues in preprocessing, (3) problems in the ML model itself. It can give you an insight into what can be improved in the datasets and/or the model. The same method can be used to compare ML models or different versions of the same model. We present the tool, the theory behind it and use cases for text-based models of various types.