New
(2026) AI Processed Human Reviewed
Cultural differences in facial features based on SHAP analysis from machine learning model estimating learners' mental states
This study evaluates whether machine learning models using facial features to estimate learners' mental states (engagement and help-seeking) can generalize across different cultures. Researchers tested Taiwanese participants using an intelligent tutoring system and compared their results to Japanese participants using LightGBM models and SHAP feature analysis. The study aims to clarify cultural similarities and differences in facial action units during online learning.
Problem
Although facial expressions are effective for tracking student engagement and help-seeking in e-learning contexts, most predictive models are created within a single culture. Because facial behaviors vary culturally, it remains unclear whether these machine learning models can generalize to international learners without performance degradation or bias. Addressing this gap is essential for developing adaptive, cross-cultural intelligent tutoring systems.
Outcome
- LightGBM models effectively classified engagement and help-seeking states for Taiwanese learners, though accuracy scores were significantly higher for Japanese participants.
- Estimating engagement relied more heavily on facial features around the eyes for Japanese participants, whereas features around the mouth were more important for Taiwanese participants.
- Help-seeking estimations demonstrated significant cultural differences in SHAP values, particularly with mouth-related features like lip stretcher (AU20) and lips part (AU25).
- Cross-cultural correlation analysis revealed that facial expressions of engagement are relatively consistent across cultures, whereas help-seeking expressions are more culturally specific.
- Estimating engagement relied more heavily on facial features around the eyes for Japanese participants, whereas features around the mouth were more important for Taiwanese participants.
- Help-seeking estimations demonstrated significant cultural differences in SHAP values, particularly with mouth-related features like lip stretcher (AU20) and lips part (AU25).
- Cross-cultural correlation analysis revealed that facial expressions of engagement are relatively consistent across cultures, whereas help-seeking expressions are more culturally specific.
What it means for you
- CIO / IT Executive: On Monday morning, direct your R&D team to prioritize exploring and piloting machine learning models that incorporate culturally-aware feature weighting for facial expression analysis in our intelligent tutoring systems, specifically investigating the differential importance of eye vs. mouth features for engagement and the impact of mouth-related Action Units (e.g., AU20, AU25) on help-seeking predictions across different cultural user bases.
- IT Manager: On Monday morning, task your data science team to begin a comprehensive review of our current intelligent tutoring system's facial recognition module, focusing on identifying and documenting the specific facial features and Action Units (AUs) it currently utilizes, and initiate a plan to evaluate the potential impact of cultural variations in these features on model performance, particularly for engagement and help-seeking, based on the provided research findings.
- Business Strategist: On Monday morning, initiate a market assessment to identify key international target markets for our intelligent tutoring systems where cultural variations in facial expressions might significantly impact user engagement and help-seeking detection, and begin preliminary discussions with the product development team about incorporating adaptive facial analysis capabilities to improve cross-cultural user experience and market penetration.
- Researcher: On Monday morning, design a controlled experiment to quantitatively measure the accuracy difference in engagement and help-seeking prediction models when using feature sets weighted for Japanese participants versus Taiwanese participants (or vice versa) on a mixed-cultural test dataset, specifically focusing on the SHAP values for mouth-related Action Units (AU20, AU25) in help-seeking scenarios.
- Policymaker: On Monday morning, draft a mandate for internal technology development guidelines that requires the evaluation and documentation of potential cultural biases in any AI models used for user interaction analysis, specifically requesting that future intelligent tutoring system deployments undergo a cross-cultural validation phase, with a focus on facial feature interpretation, before broad international rollout.