Combining adaptive local aggregation average and test-time energy adaptation for federated learning

Federated test-time adaptation (FTTA) aims to adapt knowledge from diverse source models to different but related unlabeled target data in an online and privacy-aware manner. However, existing FTTA methods struggle with decreased adaptation performance caused by data distribution shifts among clients. In this paper, we propose an FTTA method (ALAA-TTEA) called adaptive local aggregation average and test-time energy adaptation. The method consists of two continuous stages. First, in the federated aggregation stage, clients adaptively aggregate the global model and local model through adaptive local aggregation (ALA) to initialize the client model. Then, they obtain the global model through personalized training and model averaging. Second, in the test-time adaptation stage, test-time energy adaptation (TTEA) uses an energy function to transform the global model into an energy-based model. It aligns the model distribution with the test data distribution, thereby enhancing the model’s ability to adapt to the test distribution and improving overall performance. Extensive experiments demonstrate that ALAA-TTEA effectively handles data distribution shifts, including feature shift, label shift, hybrid shift, and domain shift. Moreover, it consistently outperforms existing FTTA methods under most conditions.