Research on Cross-lingual Causal Representation Learning Based on Multi-dimensional Semantic Disentanglement
Main Article Content
Keywords
semantic disentanglement, causal representation learning, cross-lingual transfer, language-invariant representation, natural language processing
Abstract
This paper focuses on several key challenges in cross-lingual natural language processing, including unstable semantic transfer, the entanglement of language-specific features, and insufficient interpretability of learned representations. To address these issues, the study introduces the necessity of combining multi-dimensional semantic disentanglement with causal representation learning. A cross-lingual causal representation learning framework is proposed, covering semantic dimension decomposition, causal factor modeling, language-invariant representation extraction, semantic intervention mechanisms, and cross-lingual transfer optimization. Finally, the paper summarizes the experimental findings and discusses the theoretical and practical significance of the proposed approach.
References
- [1] Ki, Dayeon, Park, Cheonbok, and Kim, Hyunjoong. "Mitigating semantic leakage in cross-lingual embeddings via orthogonality constraint." Proceedings of the 9th Workshop on Representation Learning for NLP (RepL4NLP-2024) (2024): 256-273.
- [2] Fukushima, Keita, Kajiwara, Tomoyuki, and Ninomiya, Takashi. "Reversible disentanglement of meaning and language representations from multilingual sentence encoders." Proceedings of the 5th Workshop on Multilingual Representation Learning (MRL 2025) (2025): 265-270.
- [3] Hua, Tianze, Yun, Tian, and Pavlick, Ellie. "mOthello: When do cross-lingual representation alignment and cross-lingual transfer emerge in multilingual models?" Findings of the Association for Computational Linguistics: NAACL 2024 (2024): 1585-1598.
- [4] Hämmerl, Katharina, Libovický, Jindřich, and Fraser, Alexander. "Understanding cross-lingual alignment—A survey." Findings of the Association for Computational Linguistics: ACL 2024 (2024): 10922-10943.
- [5] Li, Jiahuan, Huang, Shujian, Ching, Aarron, et al. "PreAlign: Boosting cross-lingual transfer by early establishment of multilingual alignment." Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (2024): 10246-10257.
- [6] Wang, Hetong, Minervini, Pasquale, and Ponti, Edoardo. “Probing the emergence of cross-lingual alignment during LLM training.” Findings of the Association for Computational Linguistics: ACL 2024 (2024): 12159-12173.
- [7] Liu, Danni, and Niehues, Jan. “Middle-layer representation alignment for cross-lingual transfer in fine-tuned LLMs.” Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (2025): 15979-15996.
- [8] Jung, Haeji, Oh, Changdae, Kang, Jooeon, et al. “Mitigating the linguistic gap with phonemic representations for robust cross-lingual transfer.” Proceedings of the Fourth Workshop on Multilingual Representation Learning (MRL 2024) (2024): 200-211.
- [9] Gui, Anchun, and Xiao, Han. “Multi-level multilingual semantic alignment for zero-shot cross-lingual transfer learning.” Neural Networks 173 (2024): 106217.
- [10] Ji, Shaoxiong, Mickus, Timothee, Segonne, Vincent, et al. “Can machine translation bridge multilingual pretraining and cross-lingual transfer learning?” Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024) (2024): 2809-2818.
- [11] Wang, Sicheng, Wu, Wenyi, and Zhang, Zibo. “NeighXLM: Enhancing cross-lingual transfer in low-resource languages via neighbor-augmented contrastive pretraining.” Findings of the Association for Computational Linguistics: EMNLP 2025 (2025): 3019-3030.
- [12] He, Zhimin, Zhang, Meishan, Zhang, Yue, et al. “Zero-shot cross-lingual document-level event causality identification with heterogeneous graph contrastive transfer learning.” Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024) (2024): 17835-17852.
- [13] Hudi, Febri, Mino, Hideya, Imamura, Kenji, et al. “Disentangling pretrained representation to leverage low-resource languages in multilingual machine translation.” Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024) (2024): 4797-4808.
- [14] Nguyen, Hoang, Le, Phong, and van Genabith, Josef. “CORI: CJKV benchmark with romanization integration—A step towards cross-lingual transfer beyond textual scripts.” Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024) (2024): 4056-4068.
- [15] Zhu, Zhaowei, Li, Xiaoyu, Zhang, Min, et al. “Code-switching can be better aligners: Advancing cross-lingual transfer through representation-level phrase exchange.” Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) (2024): 153-160.
Article Sidebar
This work is licensed under a Creative Commons Attribution 4.0 International License.