Measuring Pragmatic Influence in Large Language Model Instructions

It is not only what we ask large language models (LLMs) to do that matters, but also how we ask them. It comes from a paper posted to arXiv on 14 September 2026. Phrases like ``This is urgent'' or ``As your supervisor'' can shift model behavior without altering task content. We study this effect as pragmatic framing, contextual cues that shape directive interpretation rather than task specification. While prior work exploits such cues for prompt optimization or probes them as security vulnerabilities, pragmatic framing itself has received comparatively little attention as a target of controlled measurement in instruction following. To support its systematic study as a measurable property, we introduce a framework that combines three components: directive-framing decomposition separating framing context from task specification; a taxonomy organizing 400 instantiations of framing into 13 strategies across 4 mechanism clusters; and priority-based measurement that quantifies influence through observable shifts in directive prioritization. Evaluating five open-weight LLMs across different families and scales, we find that pragmatic framing produces systematic shifts in directive prioritization, and the effectiveness ranking of different strategies proves highly consistent across models. This reveals that susceptibility to pragmatic framing is a structured behavioral property of instruction-tuned systems. Measuring this susceptibility is a prerequisite for any deliberate response to it, and this work provides the framework to do so.

Why it counts

Phrases like ``This is urgent'' or ``As your supervisor'' can shift model behavior without altering task content.

Sources

Primary source: primary source

What is not known

This brief does not claim independent replication. Claims that appear only on X and not in the primary source stay unknown.