Gorshenin, Andrey K

Nikolaev D., Gorshenin A., Gaidamaka Y.

In today’s rapidly evolving world of mobile communications, 5G and 6G networks are leveraging millimeter-wave and sub-millimeter-wave frequencies to achieve faster speeds and higher capacities. To address the challenge of shorter coverage areas, integrated access and backhaul (IAB) technologies have been adopted, creating a dense and cost-effective network of relay nodes. This approach has the potential to significantly reduce the cost and time required for operators to transition to next-generation networks. This paper explores the operation of boundary nodes in IAB networks with half-duplex data transmission. To simulate the operation of the boundary node, we propose a mathematical model of a polling service system with an arbitrary number of queues in continuous time. This model is used to analyze the probabilistic-time characteristics of the system. We investigate delays in packet transmission in the network and their compliance with 5G network standards. The proposed model is analyzed using queueing theory, generating functions (GFs), and integral transformations such as Laplace (LT) and Laplace-Stieltjes (LST) transforms. As a result, a polling service model with an arbitrary number of queues and a cyclic service was designed, where requests are received during switching periods after the end of the service cycle. The GF, distributions, raw and central moments of the number of requests in queues, as well as LST, cumulative distribution functions (CDFs), and raw and central moments of request dwell time at the queue service phases, were derived. Additionally, a numerical analysis of round-trip time (RTT) fragment during data transmission was performed, allowing us to investigate the age of information metric.

Gorshenin A.

The paper presents for the first time a methodology for solving supervised learning problems, such as classification and regression, based on deep Gaussian mixture models (DGMMs). We use a self-supervised approach to construct a classifier as well as a semi-supervised one for a regressor. More than 20 public UCI datasets with various parameters were used for testing. It has been demonstrated that the greatest increase in classification accuracy of $$37.69\%$$ is achieved by using the ensemble of DGMM and extreme gradient boosting (XGBoost). The accuracy of this method exceeds that of the combination of GMM and SVM by $$14.51\%$$ . The DGMM regression (DGMMR) analogue of the Gaussian mixture model regression (GMMR) is introduced as a semi-supervised learning algorithm. On the test data, the best results were shown by the ensemble of DGMMR and XGBoost regression. The accuracy of this method exceeded the combination with support vector machines regression (SVR), as well as variants of GMMR with SVR and linear regression with SVR by $$3.58\%$$ , $$11.63\%$$ and $$32.78\%$$ , respectively.

Dostovalova A., Gorshenin A.

The paper introduces a probability-informed methodology for the segmentation of synthetic aperture radar (SAR) images in the case of small sample learning. It assumes that the amount of training data is limited to several hundred or thousand elements, which prevents the effective training of state-of-the-art neural network (NN) models. This is a typical problem for real SAR images whose characteristics depend significantly on the sensors used to produce them and cannot always be repeated within open available datasets. To solve this problem, we propose NN models called Probability-Informed Neural Networks (PrINNs). As part of our approach, we introduce the use of probability models as a source of additional features for data. Specifically, the training dataset is enriched by modeling the pixel brightness using a finite normal mixture. We prove that such an extension can reduce errors in the learning process theoretically. The resulting enriched dataset is segmented using attention-based convolutional NNs or visual transformers. Then, post-processing is implemented based on another probability model—quadtree, which is a special case of random Markov fields. As we have theoretically demonstrated, this part of PrINNs is analogous to the graph-convolutional NNs with fixed weights. Using open SAR images obtained by different radars (namely, Sentinel-1, Capella, ESAR and HRSID) with various types of underlying surfaces, the possibility of improving segmentation quality based on PrINNs is demonstrated. We tested various combinations of methods from the PrINNs architecture, and in all cases, the PrINN approach we proposed was superior to any other combination of these methods. From the point of view of the achieved accuracy metrics, the mean $$F_1$$ score increased up to $$19.24\%$$ , and the median $$F_1$$ score was improved up to $$9.57\%$$ . Some further architectural improvements to PrINNs are also discussed in the paper.

Dostovalova A.M., Gorshenin A.K.

Abstract The paper develops an approach to probability informing deep neural networks, that is, improving their results by using various probability models within architectural elements. We introduce factor analyzers with additive and impulse noise components as such models. The identifiability of the model is proved. The relationship between the parameter estimates by the methods of least squares and maximum likelihood is established, which actually means that the estimates of the parameters of the factor analyzer obtained within the informed block are unbiased and consistent. A mathematical model is used to create a new architectural element that implements the fusion of multiscale image features to improve classification accuracy in the case of a small volume of training data. This problem is typical for various applied tasks, including remote sensing data analysis. Various widely used neural network classifiers (EfficientNet, MobileNet, and Xception), both with and without a new informed block, are tested. It is demonstrated that on the open datasets UC Merced (remote sensing data) and Oxford Flowers (flower images), informed neural networks achieve a significant increase in accuracy for this class of tasks: the largest improvement in Top-1 accuracy was 6.67% (mean accuracy without informing equals 87.3%), while Top-5 accuracy increased by 1.49% (mean base accuracy value is 96.27%).

Dostovalova A.M., Gorshenin A.K., Starichkova J.V., Arzamasov K.M.

Data processing methods using neural networks are gaining increasing popularity in a variety of medical diagnostic problems. Most often, such methods are used in the study of medical images of human organs using CT scan and magnetic resonance imaging, ultrasound and other non-invasive research methods. Diagnosing pathology in this case is the problem of segmenting a medical image, that is, searching for groups (regions) of pixels that characterize certain objects in them. One of the most successful methods for solving this problem is the U-Net neural network architecture developed in 2015. This review examines various modifications of the classic U-Net architecture. The reviewed papers are divided into several key areas: modifications of the encoder and decoder, the use of attention blocks, combination with elements of other architectures, methods for introducing additional features, transfer learning and approaches for processing small sets of real data. Various training sets are considered, for which the best values of various metrics achieved in the literature are given (similarity coefficient Dice, intersection over union IoU, overall accuracy and some others). A summary table is provided indicating the types of images analyzed and the pathologies detected on them. Promising directions for further modifications to improve the quality of solving segmentation problems are outlined. This review can be useful for determining a set of tools for identifying various diseases, primarily cancers. The presented algorithms can be a basis of professional intelligent medical assistants.

Gorshenin A.K., Vilyaev A.L.

This paper presents a new approach in the field of probability-informed machine learning (ML). It implies improving the results of ML algorithms and neural networks (NNs) by using probability models as a source of additional features in situations where it is impossible to increase the training datasets for various reasons. We introduce connected mixture components as a source of additional information that can be extracted from a mathematical model. These components are formed using probability mixture models and a special algorithm for merging parameters in the sliding window mode. This approach has been proven effective when applied to real-world time series data for short- and medium-term forecasting. In all cases, the models informed by the connected mixture components showed better results than those that did not use them, although different informed models may be effective for various datasets. The fundamental novelty of the research lies both in a new mathematical approach to informing ML models and in the demonstrated increase in forecasting accuracy in various applications. For geophysical spatiotemporal data, the decrease in Root Mean Square Error (RMSE) was up to 27.7%, and the reduction in Mean Absolute Percentage Error (MAPE) was up to 45.7% compared with ML models without probability informing. The best metrics values were obtained by an informed ensemble architecture that fuses the results of a Long Short-Term Memory (LSTM) network and a transformer. The Mean Squared Error (MSE) for the electricity transformer oil temperature from the ETDataset had improved by up to 10.0% compared with vanilla methods. The best MSE value was obtained by informed random forest. The introduced probability-informed approach allows us to outperform the results of both transformer NN architectures and classical statistical and machine learning methods.

Gorshenin A., Kozlovskaya A., Gorbunov S., Kochetkova I.

The paper proposes an approach to the joint use of statistical and machine learning (ML) models to solve the problems of the precise reconstruction of historical events, real-time detection of ongoing incidents, and the prediction of future quality of service-related occurrences for prospective development of the modern networks. For forecasting, a regression version of the deep Gaussian mixture model (DGMM) is introduced. First, the preliminary clustering based on the finite normal mixtures is performed. This information is then used as an input for some supervised ML algorithm. It is the basic concept of the probability-informed ML approach in the field of telecommunications networks. Using the real-world datasets from a Portuguese mobile operator as well as public cellular traffic data, the article compares this approach with methods such as random forests, support vector machine regression, gradient boosting and LSTM. Vector autoregression, informed by the parameters of the generalized gamma (GG) distribution, which has also been successfully used to reconstruct past traffic patterns, is also used as a benchmark. We demonstrate that DGMM-based regression is 6.82−22.8 times faster than LSTM for the dataset. Moreover, DGMM-based regression can achieve better results for the most important traffic characteristics (average and total traffic, the number of users). For metrics MAPE and RMSE, it surpasses the results of statistical methods up to 46.7% (RMSE) and 91.5% (MAPE) (median increases are 28.0% and 80.1%, respectively), as well as for ML methods up to 13.0% (RMSE) and 35.7% (MAPE) (median increases are 0.39% and 2.5%, respectively). Thus, the use of a probability-informed approach for telecommunication data seems optimal for the computational speed and accuracy trade-off. Also, we introduce a novel statistical hypothesis testing method based on GG distribution for detecting suspected anomalies in traffic.

Kushchazli A., Safargalieva A., Kochetkova I., Gorshenin A.

The advancement of cloud computing technologies has positioned virtual machine (VM) migration as a critical area of research, essential for optimizing resource management, bolstering fault tolerance, and ensuring uninterrupted service delivery. This paper offers an exhaustive analysis of VM migration processes within cloud infrastructures, examining various migration types, server load assessment methods, VM selection strategies, ideal migration timing, and target server determination criteria. We introduce a queuing theory-based model to scrutinize VM migration dynamics between servers in a cloud environment. By reinterpreting resource-centric migration mechanisms into a task-processing paradigm, we accommodate the stochastic nature of resource demands, characterized by random task arrivals and variable processing times. The model is specifically tailored to scenarios with two servers and three VMs. Through numerical examples, we elucidate several performance metrics: task blocking probability, average tasks processed by VMs, and average tasks managed by servers. Additionally, we examine the influence of task arrival rates and average task duration on these performance measures.

Belyaev K.P., Gorshenin A.K., Korolev V.Y., Osipova A.A.

This paper compares two statistical methods for parameter reconstruction (random drift and diffusion coefficients of the Itô stochastic differential equation, SDE) in the problem of stochastic modeling of air–sea heat flux increment evolution. The first method relates to a nonparametric estimation of the transition probabilities (wherein consistency is proven). The second approach is a semiparametric reconstruction based on the approximation of the SDE solution (in terms of distributions) by finite normal mixtures using the maximum likelihood estimates of the unknown parameters. This approach does not require any additional assumptions for the coefficients, with the exception of those guaranteeing the existence of the solution to the SDE itself. It is demonstrated that the corresponding conditions hold for the analyzed data. The comparison is carried out on the simulated samples, modeling the case where the SDE random coefficients are represented in trigonometric form, which is related to common climatic models, as well as on the ERA5 reanalysis data of the sensible and latent heat fluxes in the North Atlantic for 1979–2022. It is shown that the results of these two methods are close to each other in a quantitative sense, but differ somewhat in temporal variability and spatial localization. The differences during the observed period are analyzed, and their geophysical interpretations are presented. The semiparametric approach seems promising for physics-informed machine learning models.

Gorshenin A.K., Osipova A.A., Belyaev K.P.

A dynamic stochastic model based on the Langevin stochastic differential equation is introduced for the reanalysis data of the ERA5 database to model and analyse the behavior of latent and sensible air-sea heat fluxes in the North Atlantic for the period 1979–2022. The point estimates of the random coefficients (the drift vector and the diffusion matrix) of this type of equation for the entire period under consideration are presented. The numerical methods and software tools for statistical analysis of time evolution of the coefficients as well as determination their relationships and the behavior of their maxima, averages and minima at various time intervals (days, months, years), are developed. A strong seasonality for the coefficients of the equation is demonstrated. The spatiotemporal variability of the dynamic and stochastic components of the coefficients of the Langevin equation and their relationship with jet streams of different regions of the North Atlantic is analysed. The presence of non-trivial positive trends in the drift and diffusion coefficients, especially for the latent fluxes, within the interannual variability is demonstrated. One indicates a quantitative increase in the air-sea interaction on the interannual scale. Numerical estimation was carried out using high-performance computing cluster with software implementation in the Python programming language. The tools for dynamic visualization of various quantities on geographical maps of the region under consideration are also presented.

Kochetkova I., Kushchazli A., Burtseva S., Gorshenin A.

Fifth-generation (5G) networks require efficient radio resource management (RRM) which should dynamically adapt to the current network load and user needs. Monitoring and forecasting network performance requirements and metrics helps with this task. One of the parameters that highly influences radio resource management is the profile of user traffic generated by various 5G applications. Forecasting such mobile network profiles helps with numerous RRM tasks such as network slicing and load balancing. In this paper, we analyze a dataset from a mobile network operator in Portugal that contains information about volumes of traffic in download and upload directions in one-hour time slots. We apply two statistical models for forecasting download and upload traffic profiles, namely, seasonal autoregressive integrated moving average (SARIMA) and Holt-Winters models. We demonstrate that both models are suitable for forecasting mobile network traffic. Nevertheless, the SARIMA model is more appropriate for download traffic (e.g., MAPE [mean absolute percentage error] of 11.2% vs. 15% for Holt-Winters), while the Holt-Winters model is better suited for upload traffic (e.g., MAPE of 4.17% vs. 9.9% for SARIMA and Holt-Winters, respectively).

Gorshenin A.K., Vilyaev A.L.

The paper proposes the use of related components by the method of the moving separation of mixtures as nontrivial features to expand the feature space in problems of the learning of recurrent neural networks. These features are added based on the approximation of data increments using probabilistic models based on finite normal mixtures. To take into account relationships in the data as well as in related components, the article uses the long short-term memory variant of recurrent architectures. The proposed approach is used to build an automated trading strategy based on an ensemble of the long short-term memory networks for the three most commonly traded currency pairs: euro–US dollar, US dollar–Japanese yen, and euro–pound sterling, for which data are taken from January 2011 to the end of September 2021. It is shown that the profitability of the developed ensemble long short-term memory model using additional features, i.e., information on the probabilistic distribution of data increments, outperforms both the basic methods of algorithmic trading by financial indicators (advantage of up to 32.2% on test data) and well-known approaches based on long short-term memory networks without statistical expansion of the feature space (advantage of up to 23.3%). For the best models within the framework of model trading, the final and annual yields are found to be up to 99% and 54%, respectively.

Belyaev K.P., Gorshenin A.K., Korolev V.Y., Plekhanov A.D.

Korolev V.Y., Sokolov I.A., Gorshenin A.K.

Extreme values are considered in samples with random size that have a mixed Poisson distribution being generated by a doubly stochastic Poisson process. We prove some inequalities providing bounds on the rate of convergence in limit theorems for the distributions of max-compound Cox processes.

Yang J., Huang L.

Kushchazli A., Leonteva K., Kochetkova I., Khakimov A.

The efficient migration of virtual machines (VMs) is critical for optimizing resource management, ensuring service continuity, and enhancing resiliency in cloud and edge computing environments, particularly as 6G networks demand higher reliability and lower latency. This study addresses the challenges of dynamically balancing server loads while minimizing downtime and migration costs under stochastic task arrivals and variable processing times. We propose a queuing theory-based model employing continuous-time Markov chains (CTMCs) to capture the interplay between VM migration decisions, server resource constraints, and task processing dynamics. The model incorporates two migration policies—one minimizing projected post-migration server utilization and another prioritizing current utilization—to evaluate their impact on system performance. The numerical results show that the blocking probability for the first VM for Policy 1 is 2.1% times lower than for Policy 2 and the same metric for the second VM is 4.7%. The average server’s resource utilization increased up to 11.96%. The framework’s adaptability to diverse server–VM configurations and stochastic demands demonstrates its applicability to real-world cloud systems. These results highlight predictive resource allocation’s role in dynamic environments. Furthermore, the study lays the groundwork for extending this framework to multi-access edge computing (MEC) environments, which are integral to 6G networks.

Almusfar L.A.

Karaev A.K., Borisova O.V.

The subject of the study is the choice of a model for financial forecasting of budget revenues, which allows the most correct assessment and obtaining a forecast value for the next period. The purpose of the study is to identify promising models of financial forecasting of budget revenues of the Russian Federation. DSGE models used since the 60s of the twentieth century have failed to identify a number of crises and timely predict the level of changes in government revenues in the United States, the Eurozone, and Russia, which did not allow for prompt adjustment of the policy pursued in the field of public revenue management. The novelty of the study consists in identifying the shortcomings of the modern methodology of financial forecasting associated with the obsolescence of the approaches used and the need to search for new models that allow you to quickly refine the prognostic results. The study used such methods as measuring predictive values and the size of their errors, analyzing and comparing the results obtained using methods and models of machine and deep learning. As a result of the study of predictive methods and models of machine and deep learning used in real business, the stock market and public finance, the most promising of them were selected. The main selection criteria were the possibility of modeling nonlinear relationships of parameters, the efficiency of calculation, the minimality of error, and the absence of a problem with retraining. In the course of the study, the expediency of time series decomposition was revealed, which made it possible to minimize predictive errors and choose the most accurate model for forecasting budget revenues of the Russian Federation. The results of the study can be used to form a systemof predictive indicators used to develop a dashboard system for civil servants in order to improve the accuracy and efficiency of their decisions.

Su X., Alatas B., Sohaib O.

Estimated Time of Arrival (ETA) is a crucial task in the logistics and transportation industry, aiding businesses and individuals in optimizing time management and improving operational efficiency. This study proposes a novel Graph Recurrent Neural Network (GRNN) model that integrates external factor data. The model first employs a Multilayer Perceptron (MLP)-based external factor data embedding layer to categorize and combine influencing factors into a vector representation. A Graph Recurrent Neural Network, combining Long Short-Term Memory (LSTM) and GNN models, is then used to predict ETA based on historical data. The model undergoes both offline and online evaluation experiments. Specifically, the offline experiments demonstrate a 5.3% reduction in RMSE on the BikeNYC dataset and a 6.1% reduction on the DidiShenzhen dataset, compared to baseline models. Online evaluation using Baidu Maps data further validates the model's effectiveness in real-time scenarios. These results underscore the model's potential in improving ETA predictions for urban traffic systems.

Li L., Gao X.

Internet services are increasingly being deployed using cloud computing. However, the workload of an Internet service is not constant; therefore, the required cloud computing resources need to be allocated elastically to minimize the associated costs. Thus, this study proposes a proactive cloud resource scheduling framework. First, we propose a new workload prediction method—named the adaptive two-stage multi-neural network based on long short-term memory (LSTM)—which can adaptively route prediction tasks to the corresponding LSTM sub-model according to the workload change trend (i.e., uphill and downhill categories), in order to improve the predictive accuracy. To avoid the cost associated with manual labeling of the training data, the first-order gradient feature is used with the k-means algorithm to cluster and label the original training data set automatically into uphill and downhill training data sets. Then, based on stochastic queueing theory and the proposed prediction method, a maximum cloud service profit resource search algorithm based on the network workload prediction algorithm is proposed to identify a suitable number of virtual machines (VMs) in order to avoid delays in resource adjustment and increase the service profit. The experimental results demonstrate that the proposed proactive adaptive elastic resource scheduling framework can improve the workload prediction accuracy (MAPE: 0.0276, RMSE: 3.7085, R2: 0.9522) and effectively allocate cloud resources.

Zhang Y.

Gorshenin A.

The paper presents for the first time a methodology for solving supervised learning problems, such as classification and regression, based on deep Gaussian mixture models (DGMMs). We use a self-supervised approach to construct a classifier as well as a semi-supervised one for a regressor. More than 20 public UCI datasets with various parameters were used for testing. It has been demonstrated that the greatest increase in classification accuracy of $$37.69\%$$ is achieved by using the ensemble of DGMM and extreme gradient boosting (XGBoost). The accuracy of this method exceeds that of the combination of GMM and SVM by $$14.51\%$$ . The DGMM regression (DGMMR) analogue of the Gaussian mixture model regression (GMMR) is introduced as a semi-supervised learning algorithm. On the test data, the best results were shown by the ensemble of DGMMR and XGBoost regression. The accuracy of this method exceeded the combination with support vector machines regression (SVR), as well as variants of GMMR with SVR and linear regression with SVR by $$3.58\%$$ , $$11.63\%$$ and $$32.78\%$$ , respectively.

Dodonov V., Chechurin L.

The dynamics of a three-sector economy (productions of materials, investment, consumption) is considered. The problem of optimal allocation of resources over a time interval is posed. The allocation of labor and the allocation of investment are chosen as control variables. The numerical maximization of the total per capita consumption is carried out. Per capita variables, time discretization, and equation approximations are used. An algorithm for quickly finding the optimal control is proposed. The solution for a particular case of the economy is obtained. A better result is shown with the accumulation of excess materials than with the production of only the required amount of materials.

Skrzypczak D., Gorazda K., Mikula K., Mironiuk M., Kominko H., Sawska K., Evrard D., Trzaska K., Moustakas K., Chojnacka K.

Dostovalova A., Gorshenin A.

The paper introduces a probability-informed methodology for the segmentation of synthetic aperture radar (SAR) images in the case of small sample learning. It assumes that the amount of training data is limited to several hundred or thousand elements, which prevents the effective training of state-of-the-art neural network (NN) models. This is a typical problem for real SAR images whose characteristics depend significantly on the sensors used to produce them and cannot always be repeated within open available datasets. To solve this problem, we propose NN models called Probability-Informed Neural Networks (PrINNs). As part of our approach, we introduce the use of probability models as a source of additional features for data. Specifically, the training dataset is enriched by modeling the pixel brightness using a finite normal mixture. We prove that such an extension can reduce errors in the learning process theoretically. The resulting enriched dataset is segmented using attention-based convolutional NNs or visual transformers. Then, post-processing is implemented based on another probability model—quadtree, which is a special case of random Markov fields. As we have theoretically demonstrated, this part of PrINNs is analogous to the graph-convolutional NNs with fixed weights. Using open SAR images obtained by different radars (namely, Sentinel-1, Capella, ESAR and HRSID) with various types of underlying surfaces, the possibility of improving segmentation quality based on PrINNs is demonstrated. We tested various combinations of methods from the PrINNs architecture, and in all cases, the PrINN approach we proposed was superior to any other combination of these methods. From the point of view of the achieved accuracy metrics, the mean $$F_1$$ score increased up to $$19.24\%$$ , and the median $$F_1$$ score was improved up to $$9.57\%$$ . Some further architectural improvements to PrINNs are also discussed in the paper.

Choumal A., Rizwan M., Jha S.

In recent years, integration of sustainable energy sources integration into power grids has significantly increased data influx, presenting opportunities and challenges for power system management. The intermittent nature of photovoltaic power output, coupled with stochastic charging patterns and high demands of electric vehicles, places considerable strain on system resources. Consequently, short-term forecasting of photovoltaic power output and electric vehicle charging load becomes crucial to ensuring stability and enhancing unit commitment and economic dispatch. The trends of energy transition accumulate vast data through sensors, wireless transmission, network communication, and cloud computing technologies. This paper addresses these challenges through a comprehensive framework focused on big data analytics, employing Apache Spark that is developed. Datasets from Yulara solar park and Palo Alto's electric vehicle charging data have been utilized for this research. The paper focuses on two primary aspects: short-term forecasting of photovoltaic power generation and the exploration of electric vehicle user clustering addressed using artificial intelligence. Leveraging the supervised regression and unsupervised clustering algorithms available within the PySpark library enables the execution of data visualization, analysis, and trend identification methodologies for both photovoltaic power and electric vehicle charging behaviors. The proposed analysis offers significant insights into the resilience and effectiveness of these algorithms, so enabling informed decision-making in the area of power system management.

Dostovalova A.M., Gorshenin A.K.

Abstract The paper develops an approach to probability informing deep neural networks, that is, improving their results by using various probability models within architectural elements. We introduce factor analyzers with additive and impulse noise components as such models. The identifiability of the model is proved. The relationship between the parameter estimates by the methods of least squares and maximum likelihood is established, which actually means that the estimates of the parameters of the factor analyzer obtained within the informed block are unbiased and consistent. A mathematical model is used to create a new architectural element that implements the fusion of multiscale image features to improve classification accuracy in the case of a small volume of training data. This problem is typical for various applied tasks, including remote sensing data analysis. Various widely used neural network classifiers (EfficientNet, MobileNet, and Xception), both with and without a new informed block, are tested. It is demonstrated that on the open datasets UC Merced (remote sensing data) and Oxford Flowers (flower images), informed neural networks achieve a significant increase in accuracy for this class of tasks: the largest improvement in Top-1 accuracy was 6.67% (mean accuracy without informing equals 87.3%), while Top-5 accuracy increased by 1.49% (mean base accuracy value is 96.27%).

Korolev V.Y., Shevtsova I.G., Shestakov O.V.

Fu Y., Lee X., Wei J., Li Y., Peng B.

Alkilane K., He Y., Lee D.

Jia B., Wu H., Guo K.

We explore the influence and advantages of integrating chaotic systems with deep learning for time series forecasting in this paper. It proposes a novel deep learning method based on the Chen system, which leverages the randomness, sensitivity, and diversity of chaotic mapping to enhance the performance and efficiency of deep learning models. We introduce a deep learning framework that integrates chaotic systems, providing an innovative and effective approach for time series forecasting. The research utilizes three different types of deep learning models as baselines—Long Short-Term Memory, Neural Basis Expansion Analysis, and Transformer—and compares them with their chaotic counterparts to demonstrate the impact of chaotic systems on various deep learning architectures. Experimental validation is conducted on thirteen available time series datasets, assessing the models' forecasting accuracy, runtime, and resource occupancy. The effectiveness and superiority of the chaotic deep learning method are verified across diverse datasets, including stock markets, electricity, and air quality, showcasing significant improvements over traditional model initialization methods.

Gorshenin A.K., Vilyaev A.L.

This paper presents a new approach in the field of probability-informed machine learning (ML). It implies improving the results of ML algorithms and neural networks (NNs) by using probability models as a source of additional features in situations where it is impossible to increase the training datasets for various reasons. We introduce connected mixture components as a source of additional information that can be extracted from a mathematical model. These components are formed using probability mixture models and a special algorithm for merging parameters in the sliding window mode. This approach has been proven effective when applied to real-world time series data for short- and medium-term forecasting. In all cases, the models informed by the connected mixture components showed better results than those that did not use them, although different informed models may be effective for various datasets. The fundamental novelty of the research lies both in a new mathematical approach to informing ML models and in the demonstrated increase in forecasting accuracy in various applications. For geophysical spatiotemporal data, the decrease in Root Mean Square Error (RMSE) was up to 27.7%, and the reduction in Mean Absolute Percentage Error (MAPE) was up to 45.7% compared with ML models without probability informing. The best metrics values were obtained by an informed ensemble architecture that fuses the results of a Long Short-Term Memory (LSTM) network and a transformer. The Mean Squared Error (MSE) for the electricity transformer oil temperature from the ETDataset had improved by up to 10.0% compared with vanilla methods. The best MSE value was obtained by informed random forest. The introduced probability-informed approach allows us to outperform the results of both transformer NN architectures and classical statistical and machine learning methods.

Cui Y., Li Z., Wang Y., Dong D., Gu C., Lou X., Zhang P.

Pham T., Li X., Nguyen K.

Gorshenin A., Kozlovskaya A., Gorbunov S., Kochetkova I.

The paper proposes an approach to the joint use of statistical and machine learning (ML) models to solve the problems of the precise reconstruction of historical events, real-time detection of ongoing incidents, and the prediction of future quality of service-related occurrences for prospective development of the modern networks. For forecasting, a regression version of the deep Gaussian mixture model (DGMM) is introduced. First, the preliminary clustering based on the finite normal mixtures is performed. This information is then used as an input for some supervised ML algorithm. It is the basic concept of the probability-informed ML approach in the field of telecommunications networks. Using the real-world datasets from a Portuguese mobile operator as well as public cellular traffic data, the article compares this approach with methods such as random forests, support vector machine regression, gradient boosting and LSTM. Vector autoregression, informed by the parameters of the generalized gamma (GG) distribution, which has also been successfully used to reconstruct past traffic patterns, is also used as a benchmark. We demonstrate that DGMM-based regression is 6.82−22.8 times faster than LSTM for the dataset. Moreover, DGMM-based regression can achieve better results for the most important traffic characteristics (average and total traffic, the number of users). For metrics MAPE and RMSE, it surpasses the results of statistical methods up to 46.7% (RMSE) and 91.5% (MAPE) (median increases are 28.0% and 80.1%, respectively), as well as for ML methods up to 13.0% (RMSE) and 35.7% (MAPE) (median increases are 0.39% and 2.5%, respectively). Thus, the use of a probability-informed approach for telecommunication data seems optimal for the computational speed and accuracy trade-off. Also, we introduce a novel statistical hypothesis testing method based on GG distribution for detecting suspected anomalies in traffic.

Ji C., Xu Y., Lu Y., Huang X., Zhu Y.

Giuliano R.

Li Y., Xiao L., Wei H., Kou Y., Yang L., Li D.

Accurate short-term motion prediction is essential for safe offshore operations, and various data-driven models have been developed recently for this purpose. To improve the accuracy, interpretability, and robustness of data-driven models, a novel deep-learning model embedded with the information from time–frequency analysis, named the time–frequency physics-informed (TFPI) model, is proposed to provide physics information and constraints. The measured waves and motions are input to the TFPI model to predict wave-induced motions. Semi-submersible experimental datasets are used for training and testing. Upon verification, the proposed TFPI model demonstrates a strong ability to provide accurate predictions with excellent interpretability. Facilitated by measured waves, the TFPI model successfully extends the forecast period to 2 min with an accuracy exceeding 80%. Moreover, it is robust against different noise levels, whereas the introduction of noise to the training datasets negligibly improves its generalizability. Finally, compared with the long short-term memory model, the proposed TFPI model demonstrates better prediction performance only based on historical information, particularly when the training datasets are limited. The combination of physics-based and deep-learning models is expected to significantly benefit minute-level motion predictions in a wide range of practical applications.

Peng Y., Guo Y., Hao R., Xu C.

Network traffic prediction plays a significant role in network management. Previous network traffic prediction methods mainly focus on the temporal relationship between network traffic, and used time series models to predict network traffic, ignoring the spatial information contained in traffic data. Therefore, the prediction accuracy is limited, especially in long-term prediction. To improve the prediction accuracy of the dynamic network traffic in the long term, we propose an Attention-based Spatial-Temporal Graph Network (ASTGN) model for network traffic prediction to better capture both the temporal and spatial relations between the network traffic. Specifically, in ASTGN, we exploit an encoder–decoder architecture, where the encoder encodes the input network traffic and the decoder outputs the predicted network traffic sequences, integrating the temporal and spatial information of the network traffic data through the Spatio-Temporal Embedding module. The experimental results demonstrate the superiority of our proposed method ASTGN in long-term prediction.

Feoktistov V., Nikolaev D., Gaidamaka Y., Samouylov K.

The presented research analyzes the probabilistic characteristics of the Integrated Access and Backhaul system developed by the 3GPP consortium for millimeter-wave wireless networks. This technology addresses the challenges of deploying mobile base stations including those on UAV in densely populated urban areas with high building density and the presence of both mobile and stationary radio signal blockers. The study encompasses the development of a model and scenario for IAB implementation, a GPSS simulator, and an analytical model of a system fragment represented as a polling-based queuing system. The achievable channel bandwidth, the average end-to-end delays, and the average number of packets on the individual routes are selected as the main performance metrics.

Hu X., Liu W., Huo H.

Accurate and real-time network traffic prediction is of paramount importance in the fields of network management, performance optimization, and fault diagnosis. It provides strong support for autonomous network control, network administration and network services. Therefore, we propose a novel approach for network traffic prediction, which integrates the Butterworth filter, Convolutional Neural Network and Long Short-Term Memory network(BWCL). First, this method the network traffic data to frequency domain processing, utilizing the Butterworth filter to extract its low-frequency component. The residual component is generated by subtracting the low-frequency component from the network traffic sequence. Then, CNN–LSTM prediction models are employed to capture the spatial and temporal features of the data in different frequency bands. Finally, the prediction results of the two models are linearly summed to represent the final prediction value. To validate the feasibility of the proposed model, we construct a variety of datasets with statistical features by taking the raw network traffic in single\multi-cell scenarios at two different temporal granularities: minutes and hours. In the Pytorch experimental environment, we evaluate the performance of the model using MSE, RMSE, MAE, and R2 performance metrics. The experimental results show that the prediction accuracy of the model is improved by 25% compared to the existing time series prediction models. This innovative approach provides new ideas in the field of time series forecasting, which has a broad application prospect.

Mauro M.D., Galatro G., Postiglione F., Song W., Liotta A.

Donatti A., Correa S.L., Martins J.S., Abelem A., Both C.B., Silva F., Suruagy J.A., Pasquini R., Moreira R., Cardoso K.V., Carvalho T.C.

Belyaev K.P., Gorshenin A.K., Korolev V.Y., Osipova A.A.

This paper compares two statistical methods for parameter reconstruction (random drift and diffusion coefficients of the Itô stochastic differential equation, SDE) in the problem of stochastic modeling of air–sea heat flux increment evolution. The first method relates to a nonparametric estimation of the transition probabilities (wherein consistency is proven). The second approach is a semiparametric reconstruction based on the approximation of the SDE solution (in terms of distributions) by finite normal mixtures using the maximum likelihood estimates of the unknown parameters. This approach does not require any additional assumptions for the coefficients, with the exception of those guaranteeing the existence of the solution to the SDE itself. It is demonstrated that the corresponding conditions hold for the analyzed data. The comparison is carried out on the simulated samples, modeling the case where the SDE random coefficients are represented in trigonometric form, which is related to common climatic models, as well as on the ERA5 reanalysis data of the sensible and latent heat fluxes in the North Atlantic for 1979–2022. It is shown that the results of these two methods are close to each other in a quantitative sense, but differ somewhat in temporal variability and spatial localization. The differences during the observed period are analyzed, and their geophysical interpretations are presented. The semiparametric approach seems promising for physics-informed machine learning models.