Retentive network-based time series anomaly detection in cyber-physical systems

Time series data are ubiquitous in the operation of cyber-physical systems (CPS), encompassing network traffic data, sensor measurements, and other relevant data streams. Intelligent anomaly detection methods are crucial for identifying deviations in these time series, which can enhance system maintainability and reliability. In this study, we propose an unsupervised reconstruction-based anomaly detection method utilizing retentive network (RetNet), a novel variant of Transformer. A one-dimensional convolutional neural network is employed to map the raw time series into the RetNet model space, analogous to word embedding techniques in natural language processing. The rotary position embedding introduced in RetNet can simultaneously incorporate absolute and relative positional information, which is beneficial for modeling temporal dependency in time series. Meanwhile, the multi-scale retention mechanism of RetNet facilitates the learning of informative representations of the dominant normal patterns from training data. The observed anomalies result in larger reconstruction errors, which are subsequently detected by the peaks-over-threshold (POT) method using a dynamic threshold. We evaluate the proposed method on four benchmark datasets. Experimental results demonstrate that the proposed method outperforms the best baseline by 4.07% in terms of the F1 score.