Automatic systems with word sense induction for lexical semantic change detection

Abstract

Lexical Semantic Change Detection (LSCD) is the task of identifying words that have changed their meaning over time. The semantic change can be formalized using different mathematical models. The Jensen–Shannon Distance model quantifies semantic change at the level of word senses, while the COMPARE model works directly at the level of individual word usages. Automatic LSCD systems reflect this split: some of them explicitly induce word senses by means of clustering, while others operate at the level of individual word usages without resorting to clustering. We argue that the automatic LSCD systems with word sense induction are preferable due to their interpretability, since they answer not only the question of the degree to which a word’s meaning has changed, but also the question of which word senses caused the lexical semantic change. The main counter-argument against such systems was their inferior quality. Recent studies found that the usage-level systems outperform the systems with word sense induction, even on benchmarks that themselves rely on induced senses. In this paper, we propose an automatic LSCD system with word sense induction that surpasses the existing usage-level systems in quality. We systematically evaluate each component of the systems on benchmarks for eight languages. It is shown that specialized compact models (∼350 million parameters) achieve the quality on par with human annotators in the semantic similarity estimation problem (the Word-in-Context subtask). A new metric for evaluating clustering quality is proposed, which makes it possible to find a more optimal system configuration and thereby eliminate the quality gap. As a result, we build a sense-level LSCD system that combines interpretability, quality, and computational efficiency. In this paper, we perform a systematic evaluation of each component of automatic LSCD systems on benchmarks across eight languages. In our evaluation, we find that specialized compact models (neural networks with 350M parameters) achieve human-level quality in pairwise similarity estimation (the task known as Word-in-Context, or WiC). We also separately evaluate the clustering step in WSI-based systems across multiple clustering algorithms and configurations using a custom metric tailored to the specifics of the existing datasets. By carefully tuning the individual components and analyzing computational efficiency trade-offs, we build a WSI-based LSCD system that combines interpretability, quality, and computational efficiency.