A Review of Out-of-Distribution Generalization in Text Classification Models

Main Article Content

Ximing Yao
School of Computer Science and Technology, Wuhan University of Science and Technology, Wuhan 430065, China

DOI: https://doi.org/10.70267/cai.v3n5.138151

Keywords

out-of-distribution generalization, text classification, domain generalization, robustness

Abstract

A basic premise of standard machine learning is that the training set and the test set come from the same distribution, but this premise is often broken in practice. Text classification is one of the most fundamental tasks in natural language processing (NLP). The poor robustness of its models to out-of-distribution (OOD) samples has become an important research topic. This study reviews the latest research on OOD generalization in text classification. First, it defines OOD generalization, which can be viewed from the data perspective as systematic data variance and from the model perspective as distorted features. Second, it introduces domain generalization methods, anti-shortcut feature learning, solutions for class distribution shift, and the main methods for OOD generalization of large language models. Then, it presents evaluation benchmarks and datasets used for three types of evaluation: distribution shift, compositional generalization, and class distribution shift. Finally, it identifies the limitations of existing studies and proposes future research directions.

Abstract 17 | PDF Downloads 13

References

  • [1]Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., ... & Lempitsky, V. (2016). Domain-adversarial training of neural networks. Journal of machine learning research, 17(59), 1-35.
  • [2]Li, D., Yang, Y., Song, Y. Z., & Hospedales, T. (2018, April). Learning to generalize: Meta-learning for domain generalization. In Proceedings of the AAAI conference on artificial intelligence, 32(1), 3490-3497.
  • [3]Xie, Q., Dai, Z., Hovy, E., Luong, T., & Le, Q. (2020). Unsupervised data augmentation for consistency training. Advances in neural information processing systems, 33, 6256-6268.
  • [4]Yang, L., Zhang, S., Qin, L., Li, Y., Wang, Y., Liu, H., ... & Zhang, Y. (2023, July). Glue-x: Evaluating natural language understanding models from an out-of-distribution generalization perspective. In Findings of the association for computational linguistics: ACL 2023, 12731-12750.
  • [5]Sebastiani, F. (2002). Machine learning in automated text categorization. ACM Computing Surveys, 34(1), 1-47.
  • [6]Wang, J., Lan, C., Liu, C., Ouyang, Y., Qin, T., Lu, W., ... & Yu, P. S. (2022). Generalizing to unseen domains: A survey on domain generalization. IEEE transactions on knowledge and data engineering, 35(8), 8052-8072.
  • [7]Yang, L., Song, Y., Ren, X., Lyu, C., Wang, Y., Liu, L., ... & Zhang, Y. (2023). Out-of-distribution generalization in text classification: Past, present, and future. arXiv preprint arXiv:2305.14104.
  • [8]Costache, A. V., Gheorghe, S. F., Poesina, E., Irofti, P., & Ionescu, R. T. (2026, March). Text Classification Under Class Distribution Shift: A Survey. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 4045-4060).
  • [9]Muandet, K., Balduzzi, D., & Schölkopf, B. (2013, February). Domain generalization via invariant feature representation. In International conference on machine learning (pp. 10-18). PMLR.
  • [10]Chen, J., Ding, L., Yang, Y., Di, Z., & Xiang, Y. (2024). Domain adversarial active learning for domain generalization classification. IEEE Transactions on Knowledge and Data Engineering, 37(1), 226-238.
  • [11]Arjovsky, M., Bottou, L., Gulrajani, I., & Lopez-Paz, D. (2019). Invariant risk minimization. arXiv preprint arXiv:1907.02893.
  • [12]Krueger, D., Caballero, E., Jacobsen, J. H., Zhang, A., Binas, J., Zhang, D., ... & Courville, A. (2021, July). Out-of-distribution generalization via risk extrapolation (rex). In International conference on machine learning (pp. 5815-5826). PMLR.
  • [13]Sagawa, S., Koh, P. W., Hashimoto, T. B., & Liang, P. (2020). Distributionally robust neural networks. International Conference on Learning Representations.
  • [14]Sun, B., & Saenko, K. (2016). Deep CORAL: Correlation alignment for deep domain adaptation. European Conference on Computer Vision Workshops, 443-450.
  • [15]Balaji, Y., Sankaranarayanan, S., & Chellappa, R. (2018). MetaReg: Towards domain generalization using meta-regularization. Advances in Neural Information Processing Systems, 31, 998-1008.
  • [16]Wei, J., & Zou, K. (2019). EDA: Easy data augmentation techniques for boosting performance on text classification tasks. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, 6382-6388.
  • [17]Kaushik, D., Hovy, E., & Lipton, Z. C. (2019). Learning the difference that makes a difference with counterfactually-augmented data. arXiv preprint arXiv:1909.12434.
  • [18]Wang, Z., & Culotta, A. (2021). Robustness to spurious correlations in text classification via automatically generated counterfactuals. Proceedings of the AAAI Conference on Artificial Intelligence, 35(16), 14024-14031.
  • [19]Wu, D., Wen, L., Chen, C., & Shi, Z. (2024, February). A novel counterfactual data augmentation method for aspect-based sentiment analysis. In Asian Conference on Machine Learning (pp. 1479-1493). PMLR.
  • [20]Zhang, L., Zhang, X., Hu, L., Song, D., & Nie, L. (2025, July). Dually Self-Improved Counterfactual Data Augmentation Using Large Language Model. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 5216-5227).
  • [21]Lei, T., Barzilay, R., & Jaakkola, T. (2016). Rationalizing neural predictions. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 107-117.
  • [22]Chang, S., Zhang, Y., Yu, M., & Jaakkola, T. (2020, November). Invariant rationalization. In International Conference on Machine Learning (pp. 1448-1458). PMLR.
  • [23]Yu, M., Chang, S., Zhang, Y., & Jaakkola, T. (2019, November). Rethinking cooperative rationalization: Introspective extraction and complement control. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) (pp. 4094-4103).
  • [24]Zhou, Y., & Zhu, Z. (2025). Fighting spurious correlations in text classification via a causal learning perspective. Proceedings of the 2025 Conference of the North American Chapter of the Association for Computational Linguistics. 4264–4274.
  • [25]Veitch, V., D'Amour, A., Yadlowsky, S., & Eisenstein, J. (2021). Counterfactual invariance to spurious correlations: Why and how to pass stress tests. arXiv preprint arXiv:2106.00545.
  • [26]Weston, J., Collobert, R., Sinz, F., Bottou, L., & Vapnik, V. (2006, June). Inference with the universum. In Proceedings of the 23rd international conference on Machine learning (pp. 1009-1016).
  • [27]Palatucci, M., Pomerleau, D., Hinton, G. E., & Mitchell, T. M. (2009). Zero-shot learning with semantic output codes. Advances in Neural Information Processing Systems, 22, 1410-1418.
  • [28]Pushp, P. K., & Srivastava, M. M. (2017). Train once, test anywhere: Zero-shot learning for text classification. arXiv. https://arxiv.org/abs/1712.05972
  • [29]Scheirer, W. J., Rocha, A., Micheals, R. J., & Boult, T. E. (2011). Meta-recognition: The theory and practice of recognition score analysis. IEEE transactions on pattern analysis and machine intelligence, 33(8), 1689-1695.
  • [30]Bendale, A., & Boult, T. E. (2016). Towards open set deep networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1563-1572.
  • [31]Qin, C., Chen, X., Li, J., & Zhu, H. (2026). BOLT: Benchmarking open-world learning for text classification. Findings of the Association for Computational Linguistics: ACL 2026, 13624-13658.
  • [32]Rebuffi, S. A., Kolesnikov, A., Sperl, G., & Lampert, C. H. (2017, July). icarl: Incremental classifier and representation learning. In 2017 IEEE conference on computer vision and pattern recognition (CVPR) (pp. 5533-5542). IEEE.
  • [33]Li, Z., & Hoi, S. C. H. (2017). Learning without forgetting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(12), 2935-2947.
  • [34]Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., ... & Hadsell, R. (2017). Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13), 3521-3526.
  • [35]Xu, R., & Ding, K. (2025). Large language models for anomaly and out-of-distribution detection: A survey. Findings of the Association for Computational Linguistics: NAACL 2025, 6007-6027.
  • [36]Kwon, S. M., Xu, A. S., Yaras, C., Balzano, L., & Qu, Q. (2026, May). Towards Understanding Out-of-Distribution Generalization for In-Context Learning via Low-Dimensional Subspaces. AI Stats Workshop on Causality in the Age of AI Scaling.
  • [37]Qi, Z., Luo, H., Huang, X., Zhao, Z., Jiang, Y., Fan, X., ... & Glass, J. R. (2025, May). Quantifying generalization complexity for large language models. In International Conference on Learning Representations (Vol. 2025, pp. 97091-97113).
  • [38]Sun, Y., Hu, S., Zhou, G., Zheng, K., Hajishirzi, H., Dziri, N., & Song, D. (2026). Omega: Can llms reason outside the box in math? evaluating exploratory, compositional, and transformative generalization. Advances in Neural Information Processing Systems, 38.
  • [39]Suresh, P., Stanley, J., Joseph, S., Scimeca, L., & Bzdok, D. (2026). At the Edge of Understanding: Sparse Autoencoders Trace The Limits of Transformer Generalization. arXiv preprint arXiv:2606.26396.
  • [40]Sheikh, T. U., Sinha, S., Sam, S., Stricker, D., & Afzal, M. Z. (2025, October). ZOD: Zero-Shot and Out-of-Distribution Detection Dataset for Document Images. In 2025 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) (pp. 7545-7555). IEEE.
  • [41]Arora, U., Huang, W., & He, H. (2021, November). Types of out-of-distribution texts and how to detect them. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (pp. 10687-10701).
  • [42]Baran, M., Baran, J., Wójcik, M., Zięba, M., & Gonczarek, A. (2023, July). Classical out-of-distribution detection methods benchmark in text classification tasks. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop) (pp. 119-129).
  • [43]Lake, B., & Baroni, M. (2018, July). Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks. In International conference on machine learning (pp. 2873-2882). PMLR.
  • [44]Kim, N., & Linzen, T. (2020, November). COGS: A compositional generalization challenge based on semantic interpretation. In Proceedings of the 2020 conference on empirical methods in natural language processing (emnlp) (pp. 9087-9105).
  • [45]Keysers, D., Schärli, N., Scales, N., Buisman, H., Furrer, D., Kashubin, S., ... & Bousquet, O. (2019). Measuring compositional generalization: A comprehensive method on realistic data. arXiv preprint arXiv:1912.09713.
  • [46]Li, C., Cheng, K., Hao, W., Shao, T., & Zhang, J. (2026). Survey of compositional generalization methodologies in artificial intelligence. CAAI Transactions on Intelligent Systems, 21(4), 815-833.
  • [47]Costache, A. V., Dragomir, D. N., Gheorghe, S. F., Poesina, E., Irofti, P., & Ionescu, R. T. (2026). MOSLD-Bench: Multilingual Open-Set Learning and Discovery Benchmark for Text Categorization. arXiv preprint arXiv:2601.13437.
  • [48]Larson, S., Nagy, A., Desai, S., Desai, C., Lima, N., Yuan, Y., ... & Leach, K. (2026, August). Revising RVL-CDIP: Quantifying Errors and Test-Train Overlap. In Proceedings of the 2026 ACM Symposium on Document Engineering (pp. 1-10).

Article Sidebar

Full Text
PDF
Information
Copy Publication Information
Published
Oct 29, 2026
Issue
Vol. 3 No. 5 (2026)
Section
Research Articles
Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

How to Cite

Yao, X. (2026). A Review of Out-of-Distribution Generalization in Text Classification Models. Computers and Artificial Intelligence, 3(5), 138-151. https://doi.org/10.70267/cai.v3n5.138151