Strategies for Span Labeling with Large Language Models

Generative AI & LLMs

Published: arXiv: 2601.16946v1

Authors

Danil Semin Ondřej Dušek Zdeněk Kasner

Abstract

Large language models (LLMs) are increasingly used for text analysis tasks, such as named entity recognition or error detection. Unlike encoder-based models, however, generative architectures lack an explicit mechanism to refer to specific parts of their input. This leads to a variety of ad-hoc prompting strategies for span labeling, often with inconsistent results. In this paper, we categorize these strategies into three families: tagging the input text, indexing numerical positions of spans, and matching span content. To address the limitations of content matching, we introduce LogitMatch, a new constrained decoding method that forces the model's output to align with valid input spans. We evaluate all methods across four diverse tasks. We find that while tagging remains a robust baseline, LogitMatch improves upon competitive matching-based methods by eliminating span matching issues and outperforms other strategies in some setups.

Paper Summary

Problem

Large language models (LLMs) are being increasingly used for text analysis tasks, but they struggle with a fundamental problem: how to refer to specific parts of the input text. This is a challenge because LLMs are designed to generate new text, rather than label or annotate existing text. As a result, tasks like named entity recognition, error detection, and span labeling become difficult.

Key Innovation

To address this challenge, researchers have developed three main strategies for span labeling with LLMs: tagging the input text, indexing numerical positions of spans, and matching span content. However, these methods often have limitations and inconsistencies. To overcome these issues, the researchers propose a new method called LOGITMATCH, which uses constrained decoding to force the model's output to align with valid input spans.

Practical Impact

This research has significant practical implications for text analysis tasks, where accurate and consistent labeling is crucial. By improving the performance of LLMs in span labeling tasks, this work can lead to better error detection, named entity recognition, and information extraction. Additionally, the LOGITMATCH method can be applied to other tasks that require labeling or annotating text, such as sentiment analysis or text classification.

Analogy / Intuitive Explanation

Imagine trying to describe a specific part of a map to someone who has never seen it before. You might point to the location on the map and say "this is where the city is" or "this is the river". In a similar way, LLMs need to be able to refer to specific parts of the input text in order to perform tasks like span labeling. The LOGITMATCH method is like a GPS system that helps the LLM navigate the input text and accurately identify the desired locations.

Paper Information

Categories:

cs.CL

Published Date:

arXiv ID:

2601.16946v1

Quick Actions

Back to Home