SHFL: K-Anonymity-Based Secure Hierarchical Federated Learning Framework for Smart Healthcare Systems

Islam A., Al Amin A., Shin S.Y.

This letter presents a federated learning-basd data-accumulation scheme that combines drones and blockchain for remote regions where Internet of Things devices face network scarcity and potential cyber threats. The scheme contains a two-phase authentication mechanism in which requests are first validated using a cuckoo filter, followed by a timestamp nonce. Secure accumulation is achieved by validating models using a Hampel filter and loss checks. To increase the privacy of the model, differential privacy is employed before sharing. Finally, the model is stored in the blockchain after consent is obtained from mining nodes. Experiments are performed in a proper environment, and the results confirm the feasibility of the proposed scheme.

Ali W., Din I.U., Almogren A., Guizani M., Zuair M.

The smart grid emerges as a new era of the electronic power grid. It integrates advanced sensing technologies, communications, and controlling methods that tell how electricity travels from different generation points to consumers. In order to fulfill customers' satisfaction and two way communications, a huge number of smart meters are deployed in different countries for real-time consumption and presentation of the rigorous energy usage. The privacy of industrial ecosystems may require greater attention while considering minimum network load, lower computational resources, better energy efficiency, and accuracy of data. Different research works have been done to tackle customers' privacy, but at the cost of using more computational resources, communication overhead, and hiring of a trusting third party. In this article, we have proposed a symmetric encryption scheme for industrial ecosystems in the Internet of Thing (IoT) environment. The performance evaluation and security analysis demonstrate successful user privacy and integrity with lower computational resources and communication overhead.

Sun L., Qian J., Chen X.

Training deep learning models on sensitive user data has raised increasing privacy concerns in many areas. Federated learning is a popular approach for privacy protection that collects the local gradient information instead of raw data. One way to achieve a strict privacy guarantee is to apply local differential privacy into federated learning. However, previous works do not give a practical solution due to two issues. First, the range difference of weights in different deep learning model layers has not been explicitly considered when applying local differential privacy mechanism. Second, the privacy budget explodes due to the high dimensionality of weights in deep learning models and many query iterations of federated learning. In this paper, we proposed a novel design of local differential privacy mechanism for federated learning to address the abovementioned issues. It makes the local weights update differentially private by adapting to the varying ranges at different layers of a deep neural network, which introduces a smaller variance of the estimated model weights, especially for deeper models. Moreover, the proposed mechanism bypasses the curse of dimensionality by parameter shuffling aggregation. A series of empirical evaluations on three commonly used datasets in prior differential privacy works, MNIST, Fashion-MNIST and CIFAR-10, demonstrate that our solution can not only achieve superior deep learning performance but also provide a strong privacy guarantee at the same time.

Gribbestad M., Hassan M.U., Hameed I.A., Sundli K.

Anomaly detection refers to detecting data points, events, or behaviour that do not comply with expected or normal behaviour. For example, a typical problem related to anomaly detection on an industrial level is having little labelled data and a few run-to-failure examples, making it challenging to develop reliable and accurate prognostics and health management systems for fault detection and identification. Certain machine learning approaches for anomaly detection require normal data to train, which reduces the need for historical data with fault labels, where the main task is to differentiate between normal and anomalous behaviour. Several reconstruction-based deep learning approaches are explored in this work and compared towards detecting anomalies in air compressors. Anomalies in such systems are not point-anomalies, but instead, an increasing deviation from the normal condition as the system components start to degrade. In this paper, a descriptive range of the deviation based on the reconstruction-based techniques is proposed. Most anomaly detection approaches are considered black box models, predicting whether an event should be considered an anomaly or not. This paper proposes a method for increasing the transparency and explainability of reconstruction-based anomaly detection to indicate which parts of a system contribute to the deviation from expected behaviour. The results show that the proposed methods detect abnormal behaviour in air compressors accurately and reliably and indicate why it deviates. The proposed approach is capable of detecting faults without the need for historical examples of similar faults. The proposed method for explainable anomaly detection is crucial to any prognostics and health management (PHM) system due to its purpose of detecting deviations and identifying causes.

Chen M., Poor H.V., Saad W., Cui S.

To facilitate the deployment of machine learning in resource and privacy-constrained systems such as the Internet of Things, federated learning (FL) has been proposed as a means for enabling edge devices to train a shared learning model while promoting privacy. However, Google's seminal FL algorithm requires all devices to be directly connected with a central controller, which limits its applications. In contrast, this article introduces a novel FL framework, called collaborative FL (CFL), which enables edge devices to implement FL with less reliance on a central controller. The fundamentals of this framework are developed and a number of communication techniques are proposed so as to improve CFL performance. An overview of centralized learning, Google's FL, and CFL is presented. For each type of learning, the basic architecture as well as its advantages, drawbacks, and operating conditions are introduced. Then four CFL performance metrics are presented, and a suite of communication techniques ranging from network formation, device scheduling, mobility management, to coding are introduced to optimize the performance of CFL. For each technique, future research opportunities are discussed. In a nutshell, this article showcases how CFL can be effectively implemented at the edge of large-scale wireless systems.

Liu Y., Ma Z., Liu X., Ma S., Nepal S., Deng R.H., Ren K.

Recently, Google and other 24 institutions proposed a series of open challenges towards federated learning (FL), which include application expansion and homomorphic encryption (HE). The former aims to expand the applicable machine learning models of FL. The latter focuses on who holds the secret key when applying HE to FL. For the naive HE scheme, the server is set to master the secret key. Such a setting causes a serious problem that if the server does not conduct aggregation before decryption, a chance is left for the server to access the user’s update. Inspired by the two challenges, we propose FEDXGB, a federated extreme gradient boosting (XGBoost) scheme supporting forced aggregation. FEDXGB mainly achieves the following two breakthroughs. First, FEDXGB involves a new HE based secure aggregation scheme for FL. By combining the advantages of secret sharing and homomorphic encryption, the algorithm can solve the second challenge mentioned above, and is robust to the user dropout. Then, FEDXGB extends FL to a new machine learning model by applying the secure aggregation scheme to the classification and regression tree building of XGBoost. Moreover, we conduct a comprehensive theoretical analysis and extensive experiments to evaluate the security, effectiveness, and efficiency of FEDXGB. The results indicate that FEDXGB achieves less than 1% accuracy loss compared with the original XGBoost, and can provide about 23.9% runtime and 33.3% communication reduction for HE based model update aggregation of FL.

Thrun M.C., Ultsch A.

For high-dimensional datasets in which clusters are formed by both distance and density structures (DDS), many clustering algorithms fail to identify these clusters correctly. This is demonstrated for 32 clustering algorithms using a suite of datasets which deliberately pose complex DDS challenges for clustering. In order to improve the structure finding and clustering in high-dimensional DDS datasets, projection-based clustering (PBC) is introduced. The coexistence of projection and clustering allows to explore DDS through a topographic map. This enables to estimate, first, if any cluster tendency exists and, second, the estimation of the number of clusters. A comparison showed that PBC is always able to find the correct cluster structure, while the performance of the best of the 32 clustering algorithms varies depending on the dataset.

Liu L., Zhang J., Song S.H., Letaief K.B.

Federated Learning is a collaborative machine learning framework to train a deep learning model without accessing clients’ private data. Previous works assume one central parameter server either at the cloud or at the edge. The cloud server can access more data but with excessive communication overhead and long latency, while the edge server enjoys more efficient communications with the clients. To combine their advantages, we propose a client-edge-cloud hierarchical Federated Learning system, supported with a HierFAVG algorithm that allows multiple edge servers to perform partial model aggregation. In this way, the model can be trained faster and better communication-computation trade-offs can be achieved. Convergence analysis is provided for HierFAVG and the effects of key parameters are also investigated, which lead to qualitative design guidelines. Empirical experiments verify the analysis and demonstrate the benefits of this hierarchical architecture in different data distribution scenarios. Particularly, it is shown that by introducing the intermediate edge servers, the model training time and the energy consumption of the end devices can be simultaneously reduced compared to cloud-based Federated Learning.

Chen Y., Luo F., Li T., Xiang T., Liu Z., Li J.

Machine learning models trained on sensitive real-world data promise improvements to everything from medical screening to disease outbreak discovery. In many application domains, learning participants would benefit from pooling their private datasets, training precise machine learning models on the aggregate data, and sharing the profits of using these models. Considering privacy and security concerns often prevent participants from contributing sensitive data for training, researchers proposed several techniques to achieve data privacy in federated learning systems. However, such techniques are susceptible to causative attacks, whereby malicious participants can inject false training results with the aim of corrupting the well-learned model. To end this, in this paper, we propose a new privacy-preserving federated learning scheme that guarantees the integrity of deep learning processes. Based on the Trusted Execution Environment (TEE), we design a training-integrity protocol for this scheme, in which causative attacks can be detected. Thus, each participant is compelled to execute the privacy-preserving learning algorithm of the scheme correctly. We evaluate the performance of our scheme by prototype implementations. The experimental result shows that the scheme is training-integrity and practical.

Malik S., Rouf R., Mazur K., Kontsos A.

Structural Health Monitoring (SHM), defined as the process that involves sensing, computing, and decision making to assess the integrity of infrastructure, has been plagued by data management challenges. The Industrial Internet of Things (IIoT), a subset of Internet of Things (IoT), provides a way to decisively address SHM’s big data problem and provide a framework for autonomous processing. The key focus of IIoT is operational efficiency and cost optimization. The purpose, therefore, of the IIoT approach in this investigation is to develop a framework that connects nondestructive evaluation sensor data with real-time processing algorithms on an IoT hardware/software system to provide diagnostic capabilities for efficient data processing related to SHM. Specifically, the proposed IIoT approach is comprised of three components: the Cloud, the Fog, and the Edge. The Cloud is used to store historical data as well as to perform demanding computations such as off-line machine learning. The Fog is the hardware that performs real-time diagnostics using information received both from sensing and the Cloud. The Edge is the bottom level hardware that records data at the sensor level. In this investigation, an application of this approach to evaluate the state of health of an aerospace grade composite material at laboratory conditions is presented. The key link that limits human intervention in data processing is the implemented database management approach which is the particular focus of this manuscript. Specifically, a NoSQL database is implemented to provide live data transfer from the Edge to both the Fog and Cloud. Through this database, the algorithms used are capable to execute filtering by classification at the Fog level, as live data is recorded. The processed data is automatically sent to the Cloud for further operations such as visualization. The system integration with three layers provides an opportunity to create a paradigm for intelligent real-time data quality management.

Asad M., Moustafa A., Ito T.

Artificial Intelligence (AI) has been applied to solve various challenges of real-world problems in recent years. However, the emergence of new AI technologies has brought several problems, especially with regard to communication efficiency, security threats and privacy violations. Towards this end, Federated Learning (FL) has received widespread attention due to its ability to facilitate the collaborative training of local learning models without compromising the privacy of data. However, recent studies have shown that FL still consumes considerable amounts of communication resources. These communication resources are vital for updating the learning models. In addition, the privacy of data could still be compromised once sharing the parameters of the local learning models in order to update the global model. Towards this end, we propose a new approach, namely, Federated Optimisation (FedOpt) in order to promote communication efficiency and privacy preservation in FL. In order to implement FedOpt, we design a novel compression algorithm, namely, Sparse Compression Algorithm (SCA) for efficient communication, and then integrate the additively homomorphic encryption with differential privacy to prevent data from being leaked. Thus, the proposed FedOpt smoothly trade-offs communication efficiency and privacy preservation in order to adopt the learning task. The experimental results demonstrate that FedOpt outperforms the state-of-the-art FL approaches. In particular, we consider three different evaluation criteria; model accuracy, communication efficiency and computation overhead. Then, we compare the proposed FedOpt with the baseline configurations and the state-of-the-art approaches, i.e., Federated Averaging (FedAvg) and the paillier-encryption based privacy-preserving deep learning (PPDL) on all these three evaluation criteria. The experimental results show that FedOpt is able to converge within fewer training epochs and a smaller privacy budget.

Lim W.Y., Luong N.C., Hoang D.T., Jiao Y., Liang Y., Yang Q., Niyato D., Miao C.

In recent years, mobile devices are equipped with increasingly advanced sensing and computing capabilities. Coupled with advancements in Deep Learning (DL), this opens up countless possibilities for meaningful applications, e.g., for medical purposes and in vehicular networks. Traditional cloud-based Machine Learning (ML) approaches require the data to be centralized in a cloud server or data center. However, this results in critical issues related to unacceptable latency and communication inefficiency. To this end, Mobile Edge Computing (MEC) has been proposed to bring intelligence closer to the edge, where data is produced. However, conventional enabling technologies for ML at mobile edge networks still require personal data to be shared with external parties, e.g., edge servers. Recently, in light of increasingly stringent data privacy legislations and growing privacy concerns, the concept of Federated Learning (FL) has been introduced. In FL, end devices use their local data to train an ML model required by the server. The end devices then send the model updates rather than raw data to the server for aggregation. FL can serve as an enabling technology in mobile edge networks since it enables the collaborative training of an ML model and also enables DL for mobile edge network optimization. However, in a large-scale and complex mobile edge network, heterogeneous devices with varying constraints are involved. This raises challenges of communication costs, resource allocation, and privacy and security in the implementation of FL at scale. In this survey, we begin with an introduction to the background and fundamentals of FL. Then, we highlight the aforementioned challenges of FL implementation and review existing solutions. Furthermore, we present the applications of FL for mobile edge network optimization. Finally, we discuss the important challenges and future research directions in FL.

Khatua P.K., Ramachandaramurthy V.K., Kasinathan P., Yong J.Y., Pasupuleti J., Rajagopalan A.

The availability of renewable energy sources along with the advancement of sensing and communication technologies has resulted in the sustainable operation of modern energy systems. An intelligent grid system is the integration of sensors and actuators, which enables the system to connect and exchange energy-related data from renewable sources to a computer system and end-users in a communication network. This data can be monitored in real-time with the help of the Internet of Things (IoT). However, several challenges exist in IoT, such as security, bandwidth management, interfacing interoperability, connectivity, packet loss, and data processing. In this paper, the key challenges and outstanding issues with the IoT when incorporated with energy systems are reviewed. The objective of this paper is to assess the suitability of different data transfer and communication protocols of IoT for deployment in the modern grid system. Moreover, several wireless IoT communication technologies are compared for their suitability in the multilayer network architecture and applications of energy systems.

Zhang J., Chen B., Yu S., Deng H.

Federated learning has emerged as a promising solution for big data analytics, which jointly trains a global model across multiple mobile devices. However, participants' sensitive data information may be leaked to an untrusted server through uploaded gradient vectors. To address this problem, we propose a privacy-enhanced federated learning (PEFL) scheme to protect the gradients over an untrusted server. This is mainly enabled by encrypting participants' local gradients with Paillier homomorphic cryptosystem. In order to reduce the computation costs of the cryptosystem, we utilize the distributed selective stochastic gradient descent (DSSGD) method in the local training phase to achieve the distributed encryption. Moreover, the encrypted gradients can be further used for secure sum aggregation at the server side. In this way, the untrusted server can only learn the aggregated statistics for all the participants' updates, while each individual's private information will be well-protected. For the security analysis, we theoretically prove that our scheme is secure under several cryptographic hard problems. Exhaustive experimental results demonstrate that PEFL has low computation costs while reaching high accuracy in the settings of federated learning.

Hao M., Li H., Xu G., Liu S., Yang H.

Deep learning has been applied in many areas, such as computer vision, natural language processing and emotion analysis. Differing from the traditional deep learning that collects users' data centrally, federated deep learning requires participants to train the networks on private datasets and share the training results, and hence has more gratifying efficiency and stronger security. However, it still presents some privacy issues since adversaries can deduce users' privacy from local outputs, such as gradients. While the problem of private federated deep learning has been an active research issue, the latest research findings are still inadequate in terms of security, accuracy and efficiency. In this paper, we propose an efficient and privacy-preserving federated deep learning protocol based on stochastic gradient descent method by integrating the additively homomorphic encryption with differential privacy. Specifically, users add noises to each local gradients before encrypting them to obtain the optical performance and security. Moreover, our scheme is secure to honest-but-curious server setting even if the cloud server colludes with multiple users. Besides, our scheme supports federated learning for large-scale users scenarios and extensive experiments demonstrate our scheme has high efficiency and high accuracy compared with non-private model.

Zheng G., Gong B., Guo C., Peng T., Gong M.

Al-Mulla F.M., Almulla M.A.

Arbelaez A., Climent L.

Abstract k-Anonymization is a popular approach for sharing datasets while preserving the privacy of personal and sensitive information. It ensures that each individual is indistinguishable from at least k-1 others in the anonymized dataset through data suppression or generalization, which inevitably leads to some information loss. The goal is to achieve k-anonymization with minimal information loss. This paper presents an efficient local search framework designed to address this challenge using arbitrary information loss metrics. The framework leverages anytime capabilities, allowing it to balance computation time and solution quality, thereby progressively improving the quality of the anonymized data. Our empirical evaluation shows that the proposed local search framework significantly reduces information loss compared to current state-of-the-art solutions, providing performance improvements of up to 54% and 43% w.r.t. the k-members and l-greedy heuristic solutions, the leading algorithms for large datasets. Additionally, our solution approach outperforms the Hun-garian-based solution, the best solution approach for small-size instances, by up to 4.7% on these instances.

Papadopoulos C., Kollias K., Fragulis G.F.

Federated learning (FL) is creating a paradigm shift in machine learning by directing the focus of model training to where the data actually exist. Instead of drawing all data into a central location, which raises concerns about privacy, costs, and delays, FL allows learning to take place directly on the device, keeping the data safe and minimizing the need for transfer. This approach is especially important in areas like healthcare, where protecting patient privacy is critical, and in industrial IoT settings, where moving large numbers of data is not practical. What makes FL even more compelling is its ability to reduce the bias that can occur when all data are centralized, leading to fairer and more inclusive machine learning outcomes. However, it is not without its challenges—particularly with regard to keeping the models secure from attacks. Nonetheless, the potential benefits are clear: FL can lower the costs associated with data storage and processing, while also helping organizations to meet strict privacy regulations like GDPR. As edge computing continues to grow, FL’s decentralized approach could play a key role in shaping how we handle data in the future, moving toward a more privacy-conscious world. This study identifies ongoing challenges in ensuring model security against adversarial attacks, pointing to the need for further research in this area.

Rimez D., Legay A., Macq B.

Healthcare is a domain characterized by the continuous influx of unannotated data generated by clinical workflows. The potential benefits of integrating artificial intelligence in support of clinical decisions are therefore substantial. An efficient machine learning approach should rely on a continual finetuning of the model with human expert annotations. In order to reduce the workload of the experts, Active Learning strategies are required. They involve the active participation of medical experts to provide the most informative annotations for model finetuning. The main goal of this paper is to propose an architecture which builds trust among the annotators in the process, thereby encouraging their engagement and motivation for ongoing participation in the annotation process. We provide a Secure and Anonymous Multiparty Annotation System (SAMAS) that could be used for securely annotating data in both Active Learning and Continuous annotation workflow. This architecture is based on conditional access mechanisms and communication scheme implying a Trusted Third Party to ensure both security of communications and annotators anonymity. We verified our implementation, and its compliance to a set of predefined properties, in a simplified example of distributed annotation using Spin and Linear Temporal Logic.

Majeed A., Hwang S.O.

Karagiannis S., Ntantogian C., Magkos E., Tsohou A., Ribeiro L.L.

AbstractIn modern healthcare systems, data sources are highly integrated, and the privacy challenges are becoming a paramount concern. Despite the critical importance of privacy preservation in safeguarding sensitive and private information across various domains, there is a notable deficiency of learning and training material for privacy preservation. In this research, we present a k-anonymity algorithm explicitly for educational purposes. The development of the k-anonymity algorithm is complemented by seven validation tests, that have also been used as a basis for constructing five learning scenarios on privacy preservation. The outcomes of this research provide a practical understanding of a well-known privacy preservation technique and extends the familiarity of k-anonymity and the fundamental concepts of privacy protection to a broader audience.

Gajndran S., Muthusamy R., Ravi K., ChandraUmakantham O., Marappan S.

Shen H., Wang Y., Zhang M.

With the popularity of location services and the widespread use of trajectory data, trajectory privacy protection has become a popular research area. k-anonymity technology is a common method for achieving privacy-preserved trajectory publishing. When constructing virtual trajectories, most existing trajectory k-anonymity methods just consider point similarity, which results in a large dummy trajectory space. Suppose there are n similar point sets, each consisting of m points. The size of the space is then mn. Furthermore, to choose suitable k− 1 dummy trajectories for a given real trajectory, these methods need to evaluate the similarity between each trajectory in the space and the real trajectory, leading to a large performance overhead. To address these challenges, this paper proposes a k-anonymity trajectory privacy protection method based on the similarity of sub-trajectories. This method not only considers the multidimensional similarity of points, but also synthetically considers the area between the historic sub-trajectories and the real sub-trajectories to more fully describe the similarity between sub-trajectories. By quantifying the area enclosed by sub-trajectories, we can more accurately capture the spatial relationship between trajectories. Finally, our approach generates k−1 dummy trajectories that are indistinguishable from real trajectories, effectively achieving k-anonymity for a given trajectory. Furthermore, our proposed method utilizes real historic sub-trajectories to generate dummy trajectories, making them more authentic and providing better privacy protection for real trajectories. In comparison to other frequently employed trajectory privacy protection methods, our method has a better privacy protection effect, higher data quality, and better performance.

Asad M., Shaukat S., Hu D., Wang Z., Javanmardi E., Nakazato J., Tsukada M.

This paper explores the potential for communication-efficient federated learning (FL) in modern distributed systems. FL is an emerging distributed machine learning technique that allows for the distributed training of a single machine learning model across multiple geographically distributed clients. This paper surveys the various approaches to communication-efficient FL, including model updates, compression techniques, resource management for the edge and cloud, and client selection. We also review the various optimization techniques associated with communication-efficient FL, such as compression schemes and structured updates. Finally, we highlight the current research challenges and discuss the potential future directions for communication-efficient FL.

Han L., Huang X., Li D., Zhang Y.

In the ring-architecture-based federated learning framework, security and fairness are severely compromised when dishonest clients abort the training process after obtaining useful information. To solve the problem, we propose a Ring- architecture-based Fair Federated Learning framework called RingFFL, in which we design a penalty mechanism for FL. Before the training starts in each round, all clients that will participate in the training pay deposits in a set order and record the transactions on the blockchain to ensure that they are not tampered with. Subsequently, the clients perform the FL training process, and the correctness of the models transmitted by the clients is guaranteed by the HASH algorithm during the training process. When all clients perform honestly, each client can obtain the final model, and the number of digital currencies in each client’s wallet is kept constant; otherwise, the deposits of clients who leave halfway will be compensated to the clients who perform honestly during the training process. In this way, through the penalty mechanism, all clients either obtain the final model or are compensated, thus ensuring the fairness of federated learning. The security analysis and experimental results show that RingFFL not only guarantees the accuracy and security of the federated learning model but also guarantees the fairness.