Meta-Transfer-Learning-Based Multimodal Human Pose Estimation for Lower Limbs.

Publisher: MDPI Country of Publication: Switzerland NLM ID: 101204366 Publication Model: Electronic Cited Medium: Internet ISSN: 1424-8220 (Electronic) Linking ISSN: 14248220 NLM ISO Abbreviation: Sensors (Basel) Subsets: MEDLINE

Original Publication: Basel, Switzerland : MDPI, c2000-

Lower Extremity*/physiology ; Posture*/physiology ; Machine Learning*; Humans ; Electromyography/methods ; Wearable Electronic Devices ; Algorithms ; Robotics ; Exoskeleton Device

Accurate and reliable human pose estimation (HPE) is essential in interactive systems, particularly for applications requiring personalized adaptation, such as controlling cooperative robots and wearable exoskeletons, especially for healthcare monitoring equipment. However, continuously maintaining diverse datasets and frequently updating models for individual adaptation are both resource intensive and time-consuming. To address these challenges, we propose a meta-transfer learning framework that integrates multimodal inputs, including high-frequency surface electromyography (sEMG), visual-inertial odometry (VIO), and high-precision image data. This framework improves both accuracy and stability through a knowledge fusion strategy, resolving the data alignment issue, ensuring seamless integration of different modalities. To further enhance adaptability, we introduce a training and adaptation framework with few-shot learning, facilitating efficient updating of encoders and decoders for dynamic feature adjustment in real-time applications. Experimental results demonstrate that our framework provides accurate, high-frequency pose estimations, particularly for intra-subject adaptation. Our approach enables efficient adaptation to new individuals with only a few new samples, providing an effective solution for personalized motion analysis with minimal data.

IEEE Trans Pattern Anal Mach Intell. 2014 Jul;36(7):1325-39. (PMID: 26353306)
Neuroimage. 2018 Jul 1;174:407-419. (PMID: 29578026)
Comput Biol Med. 2021 Sep;136:104649. (PMID: 34332347)
Annu Int Conf IEEE Eng Med Biol Soc. 2021 Nov;2021:6933-6936. (PMID: 34892698)
IEEE Trans Image Process. 2022;31:1559-1572. (PMID: 35077363)
IEEE Trans Neural Syst Rehabil Eng. 2020 Oct;28(10):2333-2341. (PMID: 32894718)
Front Robot AI. 2018 Feb 19;5:14. (PMID: 33500901)
IEEE Trans Pattern Anal Mach Intell. 2023 Jun;45(6):6659-6673. (PMID: 33566759)
Sensors (Basel). 2021 Apr 20;21(8):. (PMID: 33924152)
IEEE Trans Neural Syst Rehabil Eng. 2021;29:290-299. (PMID: 33378261)
IEEE Trans Pattern Anal Mach Intell. 2023 Jun;45(6):7157-7173. (PMID: 37145952)
IEEE Trans Biomed Eng. 2024 Dec;71(12):3556-3568. (PMID: 39042539)
J Ultrasound Med. 2020 Feb;39(2):361-371. (PMID: 31432552)
IEEE Trans Cybern. 2022 Aug;52(8):8128-8141. (PMID: 33531315)
Int J Biomed Comput. 1996 Apr;41(2):87-97. (PMID: 8803669)
J Neuroeng Rehabil. 2017 Jul 11;14(1):71. (PMID: 28697795)
Med Biol Eng Comput. 2020 Jan;58(1):211-225. (PMID: 31823114)
IEEE Trans Pattern Anal Mach Intell. 2021 Jan;43(1):172-186. (PMID: 31331883)
J Biomech. 1996 Sep;29(9):1223-30. (PMID: 8872282)
Front Neurosci. 2024 Mar 20;18:1306050. (PMID: 38572147)
IEEE J Transl Eng Health Med. 2020 Feb 13;8:2100310. (PMID: 32190428)
Front Neurosci. 2022 Mar 31;16:847180. (PMID: 35431778)
IEEE J Biomed Health Inform. 2023 Jan 06;PP:. (PMID: 37018609)

2024ZYD0266 Central Guidance for Local Science and Technology Development Fund Projects; 202405 National Key Laboratory of Unmanned Aerial Vehicle Technology in NPU

Keywords: human pose estimation; knowledge fusion; meta learning; multimodal; sEMG; transfer learning

Date Created: 20250317 Date Completed: 20250513 Latest Revision: 20250513

20250514

PMC11902308

10.3390/s25051613

40096488