Released as a preprint on arXiv.
The paper releases TalkFa, a Farsi dialogue benchmark with three human-reviewed datasets: 4.2K Wikipedia-grounded dialogues, 6.6K dialogues labeled for dialogue acts and emotions, and 2.1K theatrical dialogues with sentiment labels. Experiments with six LLaMA and Mistral models show that LoRA fine-tuning recovers over 90% of its gains with only 25 to 50% of the training data, and that automatic metrics and an LLM judge overrate dialogue quality compared with human evaluation.
Authors: Neda Jamshidi, Kamyar Zeinalipour, Fahimeh Akbari, Monica Bianchini, Marco Maggini, Marco Gori.
Read the paper | Code | Models and data
All the lab's language releases are listed in Language models and datasets.