TriModNet: A Hybrid View-Invariant Three-Pronged Model for Patient Activity Monitoring in Indoor Environment

Human Activity Recognition (HAR) systems play a crucial role in healthcare applications, yet their performance is often hindered by challenges such as illumination variation, occlusion, and view variation. This research addresses the view variation problem by proposing an innovative model that combines three methodologies to enhance accuracy and robustness in the recognition of patient activities. The model integrates motion history images, trajectory-based representations from both wrists (converted into 2D plots with Gaussian smoothing) and novel hand kinematic features, addressing view-related challenges by focusing on view-invariant characteristics alongside traditional joint motion features. The model is rigorously evaluated on a newly curated dataset, LNMIIT-MVAD( LNMIIT-Multiview activity dataset), showcasing its effectiveness in capturing diverse patient activities. Validation on the NTU-RGBD120 (Nanyang Technological University RGBD 120 dataset), NTU-Fall (Nanyang Technological University fall dataset), and KTH datasets highlights the model’s adaptability to different datasets. The fusion of outputs from the three methodologies through majority voting ensures a comprehensive and reliable recognition system. This research contributes to overcoming the limitations of traditional HAR systems, specifically addressing the challenge of view variation, and demonstrates promising results for real-world applications in patient care and monitoring. The proposed methodology and architecture achieved remarkable precision on various datasets: 97. 14% on LNMIIT-MVAD, 94. 28% on NTU-RGBD120, 95. 71% on KTH and 91. 14% on NTU-Fall.