From ‘Computadora’ to ‘Ordenador’: The Impact of Spanish Dialects on LLM Classification Variance

Abstract

This study investigates the impact of Spanish dialectal variation on the performance and reproducibility of Large Language Models (LLMs) in text classification tasks. Specifically, it compares Peninsular Spanish and Mexican Spanish using benchmark datasets for sentiment analysis and fake news detection. The methodology follows the CRISP-DM framework, emphasizing data normalization, dialectal adaptation, and controlled model evaluation through multiple training runs. Models such as RoBERTa were fine-tuned and tested to quantify both intra-model and inter-dialectal variance. Results show that models trained on Peninsular Spanish achieved higher stability and accuracy in sentiment analysis, while those trained on Mexican Spanish performed better in fake news detection. These differences reveal that dialectal variation significantly influences model behavior and highlight the limitations of relying solely on one Spanish variant for NLP tasks. The findings underscore the importance of developing balanced and representative datasets that capture the linguistic diversity of Spanish, ultimately contributing to fairer and more reliable LLMs for multilingual applications.