DEVELOPMENT AND APPLICATION OF A MULTIMODAL AUTOMATIC EVALUATION METHOD FOR SPOKEN ENGLISH: ENHANCING COMMUNICATION SKILLS IN INTERNATIONAL ATHLETES

Abstract

In the era of economic globalization, effective communication in English has become essential across various fields, including international sports. As athletes increasingly engage in global competitions and media interactions, the demand for proficient spoken English under diverse and often noisy environments intensifies. Traditional computer-assisted English learning tools, however, lack comprehensive functionalities for accurate pronunciation assistance and evaluation, particularly in adverse acoustic conditions. This study introduces an advanced oral English evaluation system designed to overcome these challenges through a multimodal approach. By integrating visual cues from lip movements with auditory speech data, this system provides a robust means of pronunciation error detection and correction. The unique aspect of this model lies in its use of a multimodal fusion technique that synchronizes mouth movements with corresponding phonetic sounds, enhancing pronunciation clarity and accuracy even in noisy settings. Our multimodal system not only evaluates the basic linguistic components but also considers the physical aspects of speech, such as lip extension and mouth corner movements, crucial for clear articulation. Such features are particularly valuable for athletes who must communicate effectively in high-pressure and noisy environments, such as competitive sports arenas. Experimental results demonstrate that this innovative approach significantly improves the accuracy of spoken English evaluation, offering a practical tool for athletes to enhance their communicative abilities, ensuring better interaction and performance in international sports settings.