Out-of-the-Box and into the Ditch? Multilingual Evaluation of Generic Text Extraction Tools

Adrien Barbaresi, Gaël Lejeune


Abstract
This article examines extraction methods designed to retain the main text content of web pages and discusses how the extraction could be oriented and evaluated: can and should it be as generic as possible to ensure opportunistic corpus construction? The evaluation grounds on a comparative benchmark of open-source tools used on pages in five different languages (Chinese, English, Greek, Polish and Russian), it features several metrics to obtain more fine-grained differentiations. Our experiments highlight the diversity of web page layouts across languages or publishing countries. These discrepancies are reflected by diverging performances so that the right tool has to be chosen accordingly.
Anthology ID:
2020.wac-1.2
Volume:
Proceedings of the 12th Web as Corpus Workshop
Month:
May
Year:
2020
Address:
Marseille, France
Editors:
Adrien Barbaresi, Felix Bildhauer, Roland Schäfer, Egon Stemle
Venue:
WAC
SIG:
Publisher:
European Language Resources Association
Note:
Pages:
5–13
Language:
English
URL:
https://aclanthology.org/2020.wac-1.2
DOI:
Bibkey:
Cite (ACL):
Adrien Barbaresi and Gaël Lejeune. 2020. Out-of-the-Box and into the Ditch? Multilingual Evaluation of Generic Text Extraction Tools. In Proceedings of the 12th Web as Corpus Workshop, pages 5–13, Marseille, France. European Language Resources Association.
Cite (Informal):
Out-of-the-Box and into the Ditch? Multilingual Evaluation of Generic Text Extraction Tools (Barbaresi & Lejeune, WAC 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.wac-1.2.pdf