An Empirical Evaluation of COUTA Time-Series Anomaly Detection under Different Evaluation Protocols

Main Article Content

Feiyang Teng
School of Cyber Engineering, Xidian University, Xi’an, Shaanxi 710126, China

DOI: https://doi.org/10.70267/cai.v3n5.103121

Keywords

time-series anomaly detection, COUTA, empirical evaluation, point adjustment, threshold calibration, training contamination

Abstract

Experimental conclusions in time-series anomaly detection depend not only on the model but also on threshold selection and evaluation protocols. Taking COUTA as the object, without changing the model structure, this research trains and tests on SMAP, MSL, and PSM separately, compares original point-wise evaluation, Point Adjustment (PA), and Train-Q99 alarm results in which the threshold is determined only by the 99th percentile of valid training scores, and also conducts sensitivity experiments on PSM at 5 synthetic training contamination levels. Every group of experiments employs five model random seeds and reports the mean and population standard deviation. Results of this paper show that PA-F1 on SMAP and MSL is 0.7547 and 0.8910, respectively, whereas Train-Q99 F1 is only 0.0130 and 0.0933; the Train-Q99 F1 on PSM is 0.4568, but the normal-point false positive rate reaches 0.1844. From 0% to 10% additional contamination, PSM’s Oracle Raw F1 decreases from 0.5476 to 0.4515, while PA-F1 increases from 0.8849 to 0.9470. This research establishes a traceable evaluation result set across multiple datasets and seeds, quantifies the gap between offline optimal evaluation, and fixed-threshold alarming, and reveals the directional divergence between point-wise evaluation and PA evaluation under training contamination. The results suggest that COUTA’s performance and contamination robustness are supposed to be interpreted jointly with raw scores, threshold information sources, and alarm burden. Differences in valid evaluation positions among protocols and preprocessing conditions are clarified in the main text.

Abstract 15 | PDF Downloads 10

References

  • [1]Darban, Z. Z., Webb, G. I., Pan, S., et al. (2024). Deep learning for time series anomaly detection: A survey. ACM Computing Surveys, 57 (1), Article 15. https://doi.org/10.1145/3691338
  • [2]Sánchez-Ferrera, A., Calvo, B., & Lozano, J. A. (2026). A review on self-supervised learning in time series anomaly detection: Recent advances and open challenges. ACM Computing Surveys, 58 (5), Article 119. https://doi.org/10.1145/3770575
  • [3]Paparrizos, J., Boniol, P., Liu, Q., et al. (2025). Advances in time-series anomaly detection: Algorithms, benchmarks, and evaluation measures. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2 (pp. 6151–6161). ACM. https://doi.org/10.1145/3711896.3736565
  • [4]Liu, Q., & Paparrizos, J. (2024). The elephant in the room: Towards a reliable time-series anomaly detection benchmark. In Advances in Neural Information Processing Systems (Vol. 37, pp. 108231–108261). https://doi.org/10.52202/079017-3437
  • [5]Qiu, X., Li, Z., Qiu, W., et al. (2025). TAB: Unified benchmarking of time series anomaly detection methods. Proceedings of the VLDB Endowment, 18 (9), 2775–2789. https://doi.org/10.14778/3746405.3746407
  • [6]Zhou, X., Brif, C., & Lourentzou, I. (2026). mTSBench: Benchmarking multivariate time series anomaly detection and model selection at scale. Transactions on Machine Learning Research.
  • [7]Xu, H., Wang, Y., Jian, S., et al. (2024). Calibrated one-class classification for unsupervised time series anomaly detection. IEEE Transactions on Knowledge and Data Engineering, 36 (11), 5723–5736. https://doi.org/10.1109/TKDE.2024.3393996
  • [8]Kim, S., Choi, K., Choi, H. S., et al. (2022). Towards a rigorous evaluation of time-series anomaly detection. Proceedings of the AAAI Conference on Artificial Intelligence, 36 (7), 7194–7201. https://doi.org/10.1609/aaai.v36i7.20680
  • [9]Bhattacharya, D., Mukherjee, S., Kamanchi, C., et al. (2025). Towards unbiased evaluation of time-series anomaly detector. In 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (pp. 1–5). IEEE. https://doi.org/10.1109/ICASSP49660.2025.10890568
  • [10]Sørbø, S., & Ruocco, M. (2024). Navigating the metric maze: A taxonomy of evaluation metrics for anomaly detection in time series. Data Mining and Knowledge Discovery, 38, 1027–1068. https://doi.org/10.1007/s10618-023-00988-8
  • [11]Ghorbani, R., Reinders, M. J. T., & Tax, D. M. J. (2024). PATE: Proximity-aware time series anomaly evaluation. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (pp. 872–883). ACM. https://doi.org/10.1145/3637528.3671971
  • [12]Boniol, P., Krishna, A. K., Bruel, M., et al. (2025). VUS: Effective and efficient accuracy measures for time-series anomaly detection. The VLDB Journal, 34, Article 32. https://doi.org/10.1007/s00778-025-00907-x
  • [13]Wagner, D., Nair, A., Franks, B. J., et al. (2026). Formally exploring time-series anomaly detection evaluation metrics. In Proceedings of the 29th International Conference on Artificial Intelligence and Statistics (Vol. 300, pp. 3448–3456). PMLR.
  • [14]Sun, S. H., Sankararaman, A., & Narayanaswamy, B. M. (2024). Online adaptive anomaly thresholding with confidence sequences. In Proceedings of the 41st International Conference on Machine Learning (Vol. 235, pp. 47105–47132). PMLR.
  • [15]Alnegheimish, S., He, Z., Reimherr, M., et al. (2025). M²AD: Multi-sensor multi-system anomaly detection through global scoring and calibrated thresholding. In Proceedings of the 28th International Conference on Artificial Intelligence and Statistics (Vol. 258, pp. 4384–4392). PMLR.
  • [16]Yang, Y., Zhang, C., Zhou, T., et al. (2023). DCdetector: Dual attention contrastive representation learning for time series anomaly detection. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (pp. 3033–3045). ACM. https://doi.org/10.1145/3580305.3599295
  • [17]Sun, Y., Pang, G., Ye, G., et al. (2024). Unraveling the “anomaly” in time series anomaly detection: A self-supervised tri-domain solution. In 2024 IEEE 40th International Conference on Data Engineering (pp. 981–994). IEEE. https://doi.org/10.1109/ICDE60146.2024.00080
  • [18]Fang, Y., Xie, J., Zhao, Y., et al. (2024). Temporal-frequency masked autoencoders for time series anomaly detection. In 2024 IEEE 40th International Conference on Data Engineering (pp. 1228–1241). IEEE. https://doi.org/10.1109/ICDE60146.2024.00099
  • [19]Nam, Y., Yoon, S., Shin, Y., et al. (2024). Breaking the time-frequency granularity discrepancy in time-series anomaly detection. In Proceedings of the ACM Web Conference 2024 (pp. 4204–4215). ACM. https://doi.org/10.1145/3589334.3645556
  • [20]Dai, Z., He, L., Yang, S. H., et al. (2024). SARAD: Spatial association-aware anomaly detection and diagnosis for multivariate time series. In Advances in Neural Information Processing Systems (Vol. 37, pp. 48371–48410). https://doi.org/10.52202/079017-1533
  • [21]Wu, X., Qiu, X., Li, Z., et al. (2025). CATCH: Channel-aware multivariate time series anomaly detection via frequency patching. In Proceedings of the 13th International Conference on Learning Representations.
  • [22]Yin, T., Fu, S., Zhang, Z., et al. (2025). ScatterAD: Temporal-topological scattering mechanism for time series anomaly detection. In Advances in Neural Information Processing Systems (Vol. 38, pp. 52266–52297).
  • [23]Li, M., Liu, K., Chen, H., et al. (2025). TSINR: Capturing temporal continuity via implicit neural representations for time series anomaly detection. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1 (pp. 671–682). ACM. https://doi.org/10.1145/3690624.3709266
  • [24]Zhou, Q., Pei, C., Sun, F., et al. (2025). KAN-AD: Time series anomaly detection with Kolmogorov–Arnold networks. In Proceedings of the 42nd International Conference on Machine Learning (Vol. 267, pp. 79136–79149). PMLR.
  • [25]Chen, Y., Zhang, C., Ma, M., et al. (2023). ImDiffusion: Imputed diffusion models for multivariate time series anomaly detection. Proceedings of the VLDB Endowment, 17 (3), 359–372. https://doi.org/10.14778/3632093.3632101
  • [26]Liu, C., He, S., Zhou, Q., et al. (2024). Large language model guided knowledge distillation for time series anomaly detection. In Proceedings of the 33rd International Joint Conference on Artificial Intelligence (pp. 2162–2170). IJCAI. https://doi.org/10.24963/ijcai.2024/239
  • [27]Goswami, M., Szafer, K., Choudhry, A., et al. (2024). MOMENT: A family of open time-series foundation models. In Proceedings of the 41st International Conference on Machine Learning (Vol. 235, pp. 16115–16152). PMLR.
  • [28]Mejri, N., Lopez-Fuentes, L., Roy, K., et al. (2024). Unsupervised anomaly detection in time-series: An extensive evaluation and analysis of state-of-the-art methods. Expert Systems with Applications, 256, 124922. https://doi.org/10.1016/j.eswa.2024.124922
  • [29]Sarfraz, M. S., Chen, M. Y., Layer, L., et al. (2024). Position: Quo vadis, unsupervised time series anomaly detection? In Proceedings of the 41st International Conference on Machine Learning (Vol. 235, pp. 43461–43476). PMLR.
  • [30]Huet, A., Navarro, J. M., & Rossi, D. (2022). Local evaluation of time series anomaly detection algorithms. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (pp. 635–645). ACM. https://doi.org/10.1145/3534678.3539339
  • [31]Carmona, C. U., Aubet, F. X., Flunkert, V., et al. (2022). Neural contextual anomaly detection for time series. In Proceedings of the 31st International Joint Conference on Artificial Intelligence (pp. 2843–2851). IJCAI. https://doi.org/10.24963/ijcai.2022/394
  • [32]Ho, T. K. K., & Armanfard, N. (2025). Contaminated multivariate time-series anomaly detection with spatio-temporal graph conditional diffusion models. In Proceedings of the Forty-first Conference on Uncertainty in Artificial Intelligence (Vol. 286, pp. 1710–1729). PMLR.
  • [33]Darban, Z. Z., Yang, Y., Webb, G. I., et al. (2025). DACAD: Domain adaptation contrastive learning for anomaly detection in multivariate time series. IEEE Transactions on Knowledge and Data Engineering, 37 (8), 4485–4496. https://doi.org/10.1109/TKDE.2025.3569909
  • [34]Hundman, K., Constantinou, V., Laporte, C., et al. (2018). Detecting spacecraft anomalies using LSTMs and nonparametric dynamic thresholding. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (pp. 387–395). ACM. https://doi.org/10.1145/3219819.3219845
  • [35]Abdulaal, A., Liu, Z., & Lancewicki, T. (2021). Practical approach to asynchronous multivariate time series anomaly detection and localization. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining (pp. 2485–2494). ACM. https://doi.org/10.1145/3447548.3467174
  • [36]Hundman, K., et al. (n.d.). Telemanom: A framework for using LSTMs to detect anomalies in multivariate sensor data. GitHub. Retrieved September 13, 2026, from https://github.com/khundman/telemanom
  • [37]eBay. (n.d.). RANSynCoders: Pooled Server Metrics dataset and implementation. GitHub. Retrieved September 13, 2026, from https://github.com/eBay/RANSynCoders
  • [38]THUML. (n.d.). Time-Series-Library: Dataset collection [Data set]. Hugging Face. Retrieved September 13, 2026, from https://huggingface.co/datasets/thuml/Time-Series-Library
  • [39]Mueller, A. (2025). Open challenges in time series anomaly detection: An industry perspective [Preprint]. arXiv. https://arxiv.org/abs/2502.05392
  • [40]Park, M. Y., Lee, W. J., Kim, S. T., et al. (2025). When will it fail?: Anomaly to prompt for forecasting future anomalies in time series. In Proceedings of the 42nd International Conference on Machine Learning (Vol. 267, pp. 48086–48103). PMLR.

Article Sidebar

Full Text
PDF
Information
Copy Publication Information
Published
Oct 29, 2026
Issue
Vol. 3 No. 5 (2026)
Section
Research Articles
Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

How to Cite

Teng, F. (2026). An Empirical Evaluation of COUTA Time-Series Anomaly Detection under Different Evaluation Protocols. Computers and Artificial Intelligence, 3(5), 103-121. https://doi.org/10.70267/cai.v3n5.103121