Can’t tech a joke: AI does not understand puns, study finds

Essential brief

Key facts

Large language models can detect pun structures but do not truly understand the humor behind them.

LLMs often rely on memorized examples and can be fooled by modifications that remove pun meanings.

The success rate of LLMs in identifying unfamiliar puns can drop to as low as 20%.

AI’s grasp of humor, empathy, and cultural nuance remains limited, cautioning against their use in sensitive applications.

The study underscores the current limitations of AI in interpreting complex linguistic and cultural phenomena.

Highlights

Large language models can detect pun structures but do not truly understand the humor behind them.

LLMs often rely on memorized examples and can be fooled by modifications that remove pun meanings.

The success rate of LLMs in identifying unfamiliar puns can drop to as low as 20%.

AI’s grasp of humor, empathy, and cultural nuance remains limited, cautioning against their use in sensitive applications.

Recent research conducted by experts from Cardiff University in the UK and Ca’ Foscari University of Venice in Italy has revealed significant limitations in large language models' (LLMs) ability to comprehend puns.

While these AI systems can often identify the structural elements of a pun, they fail to grasp the underlying humor or double meanings that make puns effective.

For example, when presented with the pun “I used to be a comedian, but my life became a joke,” LLMs still recognized it as a pun even when the word “joke” was replaced with “chaotic,” which removes the pun’s double meaning.

Similarly, with the pun “Long fairy tales have a tendency to dragon,” substituting “dragon” with synonyms or unrelated words did not prevent the models from falsely detecting a pun.

This suggests that LLMs rely heavily on memorization of previously encountered puns rather than true understanding.

Professor Jose Camacho Collados of Cardiff University explained that the AI models tend to fabricate justifications for why a sentence is a pun, even when the double meaning is absent, creating an illusion of comprehension.

The research also showed that when faced with unfamiliar wordplay, the success rate of LLMs in correctly identifying puns can plummet to as low as 20%.

Another example tested was “Old LLMs never die, they just lose their attention,” which the AI still interpreted as a pun when “attention” was replaced with “ukulele,” based on a phonetic similarity to “you-kill-LLM.” Despite the apparent creativity, the AI did not genuinely understand the humor.

The findings highlight the fragile grasp LLMs have on humor, empathy, and cultural nuances, cautioning users against relying on these models for tasks requiring such understanding.

The study was presented at the 2025 Conference on Empirical Methods in Natural Language Processing in Suzhou, China, and is detailed in the paper titled "Pun unintended: LLMs and the illusion of humor understanding."