Abstract
In-bed pose estimation has shown value in fields such as hospital patient monitoring, sleep studies, and smart homes. In this paper, we explore different strategies for detecting body pose from highly ambiguous pressure data, with the aid of pre-existing pose estimators. We examine the performance of pre-trained pose estimators by using them either directly or by re-training them on two pressure datasets. We also explore other strategies utilizing a learnable pre-processing domain adaptation step, which transforms the vague pressure maps to a representation closer to the expected input space of common purpose pose estimation modules. Accordingly, we used a fully convolutional network with multiple scales to provide the pose-specific characteristics of the pressure maps to the pre-trained pose estimation module. Our complete analysis of different approaches shows that the combination of learnable pre-processing module along with re-training pre-existing image-based pose estimators on the pressure data is able to overcome issues such as highly vague pressure points to achieve very high pose estimation accuracy.
Similar content being viewed by others
References
Achilles F, Ichim AE, Coskun H, Tombari F, Noachtar S, Navab N (2016) Patient mocap: human pose estimation under blanket occlusion for hospital monitoring applications. In: International conference on medical image computing and computer-assisted intervention, pp 491–499
Andriluka M, Pishchulin L, Gehler P, Schiele B (2014) 2d human pose estimation: New benchmark and state of the art analysis. In: IEEE conference on computer Vision and Pattern Recognition (CVPR), pp 3686–3693
Cao Z, Simon T, Wei SE, Sheikh Y (2017) Realtime multi-person 2d pose estimation using part affinity fields. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 1302–1310
Casas L, Navab N, Demirci S (2019) Patient 3d body pose estimation from pressure imaging. Int J Comput Assist Radiol Surg 14(3):517–524
Chen K, Gabriel P, Alasfour A, Gong C, Doyle WK, Devinsky O, Friedman D, Dugan P, Melloni L, Thesen T et al (2018a) Patient-specific pose estimation in clinical environments. IEEE J Transl Eng Health Med 1
Chen Y, Shen C, Wei XS, Liu L, Yang J (2017) Adversarial posenet: A structure-aware convolutional network for human pose estimation. In: Proceedings of the IEEE international conference on computer vision (ICCV), pp 1212–1221
Chen Y, Wang Z, Peng Y, Zhang Z, Yu G, Sun J (2018b) Cascaded pyramid network for multi-person pose estimation. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 7103–7112
Chu X, Yang W, Ouyang W, Ma C, Yuille AL, Wang X (2017) Multi-context attention for human pose estimation. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 5669–5678
Clever HM, Kapusta A, Park D, Erickson Z, Chitalia Y, Kemp CC (2018) 3d human pose estimation on a configurable bed from a pressure image. In: IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 54–61
Clever HM, Erickson Z, Kapusta A, Turk G, Liu K, Kemp CC (2020) Bodies at rest: 3d human pose and shape estimation from a pressure image using synthetic data. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 6215–6224
Cunha JPS, Choupina HMP, Rocha AP, Fernandes JM, Achilles F, Loesch AM, Vollmar C, Hartl E, Noachtar S (2016) Neurokinect: a novel low-cost 3d video-eeg system for epileptic seizure motion quantification. PLoS One 11(1):e0145669
Davoodnia V, Etemad A (2019) Identity and posture recognition in smart beds with deep multitask learning. In: IEEE international conference on systems, man and cybernetics (SMC), pp 3054–3059
Davoodnia V, Slinowsky M, Etemad A (2020) Deep multitask learning for pervasive bmi estimation and identity recognition in smart beds. J Ambient Intell Humaniz Comput :1–15
Davoodnia V, Ghorbani S, Etemad A (2021) In-bed pressure-based pose estimation using image space representation learning. In: IEEE international conference on acoustics, speech and signal processing (ICASSP)
Goldberger AL, Amaral LA, Glass L, Hausdorff JM, Ivanov PC, Mark RG, Mietus JE, Moody GB, Peng CK, Stanley HE (2000) Physiobank, physiotoolkit, and physionet: components of a new research resource for complex physiologic signals. Circulation 101(23):e215–e220
Goldberger AL, Amaral LAN, Glass L, Hausdorff JM, Ivanov PC, Mark RG, Mietus JE, Moody GB, Peng CK, Stanley HE (2000) PhysioBank, PhysioToolkit, and PhysioNet: components of a new research resource for complex physiologic signals. Circulation 101(23):e215–e220. https://doi.org/10.1161/01.CIR.101.23.e215, http://circ.ahajournals.org/content/101/23/e215
Grimm R, Sukkau J, Hornegger J, Greiner G (2011) Automatic patient pose estimation using pressure sensing mattresses. In: Bildverarbeitung für die Medizin. Springer, pp 409–413
He K, Cao X, Shi Y, Nie D, Gao Y, Shen D (2018) Pelvic organ segmentation using distinctive curve guided fully convolutional networks. IEEE Trans Med Imaging 38(2):585–595
Hu H, Gu J, Zhang Z, Dai J, Wei Y (2018) Relation networks for object detection. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 3588–3597
Imabuchi T, Prima ODA, Ito H (2018) Automated assessment of nonverbal behavior of the patient during conversation with the healthcare worker using a remote camera. eTELEMED 2018. p 29
Insafutdinov E, Pishchulin L, Andres B, Andriluka M, Schiele B (2016) Deepercut: A deeper, stronger, and faster multi-person pose estimation model. In: European conference on computer vision (ECCV). Springer, pp 34–50
Insafutdinov E, Andriluka M, Pishchulin L, Tang S, Levinkov E, Andres B, Schiele B (2017) Arttrack: articulated multi-person tracking in the wild. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 1293–1301
Javaid AQ, Gupta R, Mihalidis A, Etemad SA (2017) Balance-based time-frequency features for discrimination of young and elderly subjects using unsupervised methods. In: IEEE EMBS international conference on biomedical & health informatics (BHI), pp 453–456
Ke L, Chang MC, Qi H, Lyu S (2018) Multi-scale structure-aware network for human pose estimation. In: Proceedings of the european conference on computer vision (ECCV), pp 713–728
Koppula HS, Saxena A (2016) Anticipating human activities using object affordances for reactive robotic response. IEEE Trans Pattern Anal Mach Intell 38(1):14–29
Lee CH, Kim DK, Kim SY, Rhee CS, Won TB (2015) Changes in site of obstruction in obstructive sleep apnea patients according to sleep position: a dise study. Laryngoscope 125(1):248–254
Lee J, Kwon H, Seo J, Shin S, Koo JH, Pang C, Son S, Kim JH, Jang YH, Kim DE et al (2015) Conductive fiber-based ultrasensitive textile pressure sensor for wearable electronics. Adv Mater 27(15):2433–2439
Lin F, Song C, Xu X, Cavuoto L, Xu W (2017) Patient handling activity recognition through pressure-map manifold learning using a footwear sensor. Smart Health 1:77–92
Liu JJ, Huang MC, Xu W, Sarrafzadeh M (2014) Bodypart localization for pressure ulcer prevention. In: Annual international conference of the ieee engineering in medicine and biology society (EMBC), pp 766–769
Liu S, Ostadabbas S (2017) A vision-based system for in-bed posture tracking. In: Proceedings of the IEEE international conference on computer vision workshops, pp 1373–1382
Liu S, Ostadabbas S (2019) Seeing under the cover: A physics guided learning approach for in-bed pose estimation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 236–245
Liu S, Yin Y, Ostadabbas S (2019) In-bed pose estimation: Deep learning with shallow dataset. IEEE J Transl Eng Health Med 7:1–12
McCabe SJ, Xue Y (2010) Evaluation of sleep position as a potential cause of carpal tunnel syndrome: preferred sleep position on the side is associated with age and gender. Hand 5(4):361–363
McCabe SJ, Gupta A, Tate DE, Myers J (2011) Preferred sleep position on the side is associated with carpal tunnel syndrome. Hand 6(2):132–137
Ostadabbas S, Pouyan MB, Nourani M, Kehtarnavaz N (2014) In-bed posture classification and limb identification. In: IEEE biomedical circuits and systems conference, pp 133–136
Rhodin H, Salzmann M, Fua P (2018) Unsupervised geometry-aware representation for 3d human pose estimation. In: Proceedings of the european conference on computer vision (ECCV), pp 750–767
Ruvalcaba-Cardenas AD, Scoleri T, Day G (2018) Object classification using deep learning on extremely low-resolution time-of-flight data. In: IEEE digital image computing: techniques and applications (DICTA), pp 1–7
Singh V, Ma K, Tamersoy B, Chang YJ, Wimmer A, O’Donnell T, Chen T (2017) Darwin: Deformable patient avatar representation with deep image network. In: International conference on medical image computing and computer-assisted intervention, pp 497–504
Tang W, Yu P, Wu Y (2018) Deeply learned compositional models for human pose estimation. In: European conference on computer vision (ECCV), pp 190–206
Tulyakov S, Liu MY, Yang X, Kautz J (2018) Mocogan: Decomposing motion and content for video generation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1526–1535
Wang CW, Hunter A (2010) Robust pose recognition of the obscured human body. Int J Comput Vis (ICCV) 90(3):313–330
Wei SE, Ramakrishna V, Kanade T, Sheikh Y (2016) Convolutional pose machines. In: IEEE conference on computer vision and pattern recognition, pp 4724–4732
Woo KY, Sears K, Almost J, Wilson R, Whitehead M, VanDenKerkhof EG (2017) Exploration of pressure ulcer and related skin problems across the spectrum of health care settings in ontario using administrative data. Int Wound J 14(1):24–30
Xiao Y, Zijie Z (2020) Infrared image extraction algorithm based on adaptive growth immune field. Neural Process Lett 51(3):2575–2587
Yang W, Li S, Ouyang W, Li H, Wang X (2017) Learning feature pyramids for human pose estimation. In: IEEE international conference on computer vision (ICCV), pp 1290–1299
Zhao ZQ, Zheng P, Xu St, Wu X (2019) Object detection with deep learning: A review. IEEE Trans Neural Netw Learn Syst 30(11):3212–3232
Zhou P, Ni B, Geng C, Hu J, Xu Y (2018) Scale-transferrable object detection. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 528–537
Acknowledgements
The Titan XP GPU used for this research was donated by the NVIDIA Corporation.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interests
The authors declare that they have no conflict of interest.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Davoodnia, V., Ghorbani, S. & Etemad, A. Estimating pose from pressure data for smart beds with deep image-based pose estimators. Appl Intell 52, 2119–2133 (2022). https://doi.org/10.1007/s10489-021-02418-y
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-021-02418-y