Research on the Length Controllability of Chinese Large Language Model Outputs Based on Post-Processing Correction
Main Article Content
Keywords
large language models, length control, instruction following, post-processing correction, controllable text generation
Abstract
Large language models often deviate from the target word count when following length constraints such as “generate X words.” Existing studies mostly focus on content quality, while paying relatively less attention to format control, especially the accuracy of length control. To address this issue, this study builds a Chinese text generation benchmark covering 5 topics and 4 word-count levels, and evaluates three representative models: DeepSeek-V3, Tongyi Qianwen (Qwen2.5), and Kimi. Based on this, a lightweight post-processing framework, LAPP (Length-Aware Post-Processing), is proposed. When the output is too long, LAPP uses sentence-level intelligent truncation; when the output is too short, it calls Qwen2.5-1.5B-Instruct for semantics-preserving expansion, and controls the length error within ±10% through iterative correction. The experimental results show that the original average deviation rates of the three models are 11.93%–38.70%, which are reduced to about 5% after LAPP correction, and the pass rate increases from 20%–70% to 85%–100%. The semantic preservation results show an average BERTScore of 0.97 between the corrected and original texts, suggesting that the framework improves length control while largely keeping the original meaning. This study shows that the “rough generation + precise correction” approach is effective for word-count control, and offers a low-cost, transferable way to improve the instruction-following ability of large language models.
References
- [1]Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., ... & Lowe, R. (2022). Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35, 27730-27744. https://doi.org/10.52202/068431-2011
- [2]Wang, J., & Sun, Y. (2024). A review of controllable text generation techniques [In Chinese]. Journal of Chinese Information Processing, 38(10).
- [3]Luo, W., & Wang, H. (2024). A review of large language model evaluation [In Chinese]. Journal of Chinese Information Processing, 38(1), 1-23..
- [4]Zhang, W., Zhang, Y., et al. (2024). An indicator system for evaluating the generation quality of Chinese large language models [In Chinese]. Journal of Chinese Information Processing, 38(3), 1-15.
- [5]Liu, K., Qu, X., Tan, F., et al. (2024). A review of large language model alignment research [In Chinese]. Telecommunications Science, 40(6), 173-194.
- [6]Zhou, J., Lu, T., Mishra, S., Brahma, S., Basu, S., Luan, Y., Zhou, D., & Hou, L. (2023). Instruction-following evaluation for large language models. https://doi.org/10.48550/arXiv.2311.07911
- [7]Ma, Q., Zhang, J., & Liu, Q. (2018). A machine translation automatic evaluation method based on fusion strategy [In Chinese]. Journal of Chinese Information Processing, 32(9), 11-19.
- [8]Bastan, M., Surdeanu, M., & Balasubramanian, N. (2023). Neurostructural decoding: Neural text generation with structural constraints. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (pp. 9496-9510). Association for Computational Linguistics. https://doi.org/10.18653/v1/2023.acl-long.528
- [9]Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 30, 5998-6008. https://doi.org/10.48550/arXiv.1706.03762
- [10]Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 4171-4186). Association for Computational Linguistics. https://doi.org/10.18653/v1/N19-1423
- [11]Kikuchi, Y., Neubig, G., Sasano, R., et al. (2016). Controlling output length in neural encoder-decoders. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (pp. 1328-1338). Association for Computational Linguistics. https://doi.org/10.18653/v1/D16-1140
- [12]Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., & Artzi, Y. (2020). BERTScore: Evaluating text generation with BERT. International Conference on Learning Representations. https://doi.org/10.48550/arXiv.1904.09675
Article Sidebar
This work is licensed under a Creative Commons Attribution 4.0 International License.