The rapid advancement in educational technology has ushered in an era where multimodal learning environments are increasingly recognized as crucial for fostering students’ cognitive and emotional development. This research explores the dual impact of these environments, integrating diverse sensory information, including visual and auditory inputs, thus offering enriched learning experiences with potential influences on students’ cognitive and emotional states. A notable gap in current research is the lack of a comprehensive understanding of how multimodal learning environments distinctly affect these aspects of student development. Traditional methodologies in this field often fail to account for the dynamic nature and individual variances inherent in the learning process, thereby limiting the depth of insights into the influence mechanisms of these environments. To bridge these gaps, this study employs a dual-methodological approach. Firstly, a sophisticated correlation matrix is developed to diagnose cognitive development within multimodal learning environments, effectively identifying key factors in cognitive processes. Secondly, an innovative emotion recognition model is utilized, incorporating sensor feature embedding and temporal feature embedding techniques, to adeptly capture students’ emotional fluctuations within these settings. The findings from this study provide substantial implications for educational practice. They enable educators to enhance the design and optimization of learning environments, promoting not only cognitive growth but also emotional well-being among students. This approach underscores the importance of addressing both cognitive and emotional domains in educational settings to foster holistic development. This study innovatively establishes a matrix correlating students, learning items, and knowledge points, facilitating a joint analysis to diagnose the cognitive development mechanism within multimodal learning environments. Additionally, an emotion recognition model, combining both sensor and temporal feature embedding networks, has been constructed. This model accurately identifies and analyzes students’ emotional states. The primary contribution lies in providing a comprehensive method for evaluating both cognitive and emotional states of students, which aids in optimizing personalized learning strategies and enhancing educational outcomes.