Published in the Proceedings of the Eleventh Italian Conference on Computational Linguistics (CLiC-it 2025), Cagliari, Italy, CEUR Workshop Proceedings.
The paper measures the difficulty of Italian crossword clues as the surprisal of the answer given the clue, computed with three language models (Llama-3-8B, Llama-2-7B and Ita-GPT-2-121M), and compares it with 60 human solvers on 160 clues. Surprisal correlated negatively with human accuracy (r = -0.62 for nominal clues), so it can serve as an automatic difficulty measure for puzzle generation.
Authors: Tommaso Iaquinta, Asya Zanollo, Achille Fusco, Kamyar Zeinalipour, Cristiano Chesi.
Read the paper | Code | Models and data
All the lab's language releases are listed in Language models and datasets.