DEEP LEARNING APPROACHES FOR HUMAN ACTIVITY RECOGNITION USING WEARABLE TECHNOLOGY
Sažetak
The need for long-term monitoring of individuals in their natural environment has initiated the development of a various number of wearable healthcare sensors for a wide range of applications: medical monitoring in clinical or home environments, physical activity assessment of athletes and recreators, baby monitoring in maternity hospitals and homes etc. Neural networks (NN) are data-driven type of modelling. NN learn from experience, without knowledge about the model of phenomenon, but knowing the desired „output” data for the training „input” data. The most promising concept of machine learning that involves NN is deep learning (DL) approach. The focus of this review is on approaches of DL for physiological activity recognition or human movement analysis purposes using wearable technologies. This review shows that deep learning techniques are useful tools for health condition prediction or overall monitoring of data streamed by wearable systems. Despite the considerable progress and wide field of applications, there are still some limitations and room for improvement of DL approaches for wearable healthcare systems which may lead to more robust and reliable technology for personalized healthcare.
Reference
Rodgers MM, Pai VM, Conroy RS. Recent advances in wearable sensors for health monitoring. IEEE Sensors Journal, 2015; 15(6): 3119-3126.
Bonato P. Wearable sensors and systems. IEEE Engineering in Medicine and Biology Magazine. 2010; 29(3): 25-36.
Vukićević S, Stamenković Z, Murugesan S, Bogdanović Z, Radenković B. A new telerehabilitation system based on internet of things. Facta Universitatis, Series: Electronics and Energetics, 2015; 29(3): 395-405.
Bloss, R. Wearable sensors bring new benefits to continuous medical monitoring, real time physical activity assessment, baby monitoring and industrial applications. Sensor Review, 2015; 35(2): 141-145.
Zheng YL, Ding XR, Poon CCY, Lo BPL, Zhang H, Zhou XL et al. Unobtrusive sensing and wearable devices for health informatics. IEEE Transactions on Biomedical Engineering, 2014; 61(5): 1538-1554.
Stoppa M, Chiolerio A. Wearable electronics and smart textiles: a critical review. Sensors. 2014; 14(7): 11957-11992.
Kumari P, Mathew L, Syal P. Increasing trend of wearables and multimodal interface for human activity monitoring: A review. Biosensors and Bioelectronics. 2017; 90: 298-307.
Paradiso R, Bianchi AM, Lau K, Scilingo EP. PSYCHE: Personalised monitoring systems for care in mental health. In: Annual international conference of the IEEE Engineering in Medicine and Biology Society (EMBC). August 31-September 4, Buenos Aires, Argentina. 2010; p. 3602-3605.
Scilingo EP, Gemignani A, Paradiso R, Taccini N, Ghelarducci B, De Rossi D. Performance evaluation of sensing fabrics for monitoring physiological and biomechanical variables. IEEE Transactions on information technology in biomedicine. 2005; 9(3): 345-352.
Liu Y, Jiang X, Cao T, Wan F, Mak PU, Mak PI et al. Implementation of SSVEP based BCI with Emotiv EPOC. In: IEEE International Conference on Virtual Environments Human-Computer Interfaces and Measurement Systems (VECIMS). 2-4 July, Tianjin, China, 2017; p. 34-37.
Taylor GS, Schmidt C. Empirical evaluation of the Emotiv EPOC BCI headset for the detection of mental actions. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting. 22-26 October, Boston, Massachusetts, USA, 2012; 56(1): 193-197.
Duvinage M, Castermans T, Petieau M, Hoellinger T, Cheron G, Dutoit T. Performance of the Emotiv Epoc headset for P300-based applications. Biomedical engineering online. 2013; 12(1): 56.
Badcock NA, Mousikou P, Mahajan Y, De Lissa P, Thie J, McArthur G. Validation of the Emotiv EPOC® EEG gaming system for measuring research quality auditory ERPs. PeerJ. 2013; 1: e38.
Ramirez R, Vamvakousis Z. Detecting emotion from EEG signals using the emotive epoc device. In: International Conference on Brain Informatics. December 4-7, Macao, China, 2012; p. 175-184.
Pham TD, Tran D. Emotion recognition using the emotiv epoc device. In: International Conference on Neural Information Processing. 12-15 November, Doha, Qatar, 2012; p. 394-399.
Lu Z, Chen X, Li Q, Zhang X, Zhou P. A hand gesture recognition framework and wearable gesture-based interaction prototype for mobile devices. IEEE transactions on human-machine systems. 2014; 44(2): 293-299.
Milica Djurić-Jovičić. Inertial Sensors Signal Processing Methods For Gait Analysis Of Patients With Impaired Gait Patterns, PhD thesis, University of Belgrade – School of Electrical Engineering, Belgrade, 2012.
Torniainen J, Cowley B, Henelius A, Lukander K, Pakarinen S. Feasibility of an electrodermal activity ring prototype as a research tool. In: 37th Annual International Conference of the Engineering in Medicine and Biology Society (EMBC), 25-29 August, Milano, Italy, 2015; p. 6433-6436.
Suoja K, Liukkonen J, Jussila, J, Salonius H, Venho N, Sillanpää V et al. Application for pre-processing and visualization of electrodermal activity wearable data. In: EMBEC & NBC. 11-15 June, Tampere, Finland, 2017; p. 93-96.
Bulling A, Gellersen H. Toward mobile eye-based human-computer interaction. IEEE Pervasive Computing, 2010; 9(4): 8-12.
Cappon G, Acciaroli G, Vettoretti M, Facchinetti A, Sparacino G. Wearable Continuous Glucose Monitoring Sensors: A Revolution in Diabetes Treatment. Electronics. 2017; 6(3): 65.
Zheng YL, Yan BP, Zhang YT, Poon CC. An armband wearable device for overnight and cuff-less blood pressure measurement. IEEE transactions on biomedical engineering. 2014; 61(7): 2179-2186.
Hayashi C. What is Data Science? Fundamental Concepts and a Heuristic Example. In: Data Science, Classification, and Related Methods. Studies in Classification, Data Analysis, and Knowledge Organization. Toky: Springer; 1998.
Bishop C. Pattern recognition and machine learning. Berlin: Springer; 2006.
Goodfellow I, Bengio Y, Courville A. Deep Learning. Cambridge:The MIT Press; 2016.
Chin-Teng L, George Lee CS. Feedforward neural networks - Neural Fuzzy Systems: A Neuro-Fuzzy Synergism to Intelligent Systems. New York: Prentice Hall; 1996.
Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems. 3-8 December, Lake Tahoe, Harrahs and Harveys, 2012; p. 1097-1105.
Lipton ZC, Berkowitz J, Elkan C. A critical review of recurrent neural networks for sequence learning. arXiv preprint arXiv:1506.00019. 2015.
Redmond SJ, Lovell NH, Yang GZ, Horsch A, Lukowicz P, Murrugarra L, Marschollek M. What does big data mean for wearable sensor systems?: Contribution of the IMIA wearable sensors in healthcare WG. Yearbook of medical informatics, 2014; 9(1): 135-142.
Hansen MM, Miron-Shatz T, Lau AYS, Paton C. Big data in science and healthcare: a review of recent literature and perspectives: contribution of the IMIA social media working group. Yearbook of medical informatics, 2014; 9(1): 21-26.
Andreu-Perez J, Leff DR, Ip HM, Yang GZ. From wearable sensors to smart implants-–toward pervasive and personalized healthcare. IEEE Transactions on Biomedical Engineering. 2015; 62(12): 2750-2762.
Simpao AF, Ahumada LM, Gálvez JA, Rehman MA. A review of analytics and clinical informatics in health care. Journal of medical systems. 2014; 38(4): 1-7.
Gravina R, Alinia P, Ghasemzadeh H, Fortino G. Multi-sensor fusion in body sensor networks: State-of-the-art and research challenges. Information Fusion. 2017; 35: 68-80.
Banaee H, Ahmed MU, Loutfi A. Data mining for wearable sensors in health monitoring systems: a review of recent trends and challenges. Sensors. 2013; 13(12): 17472-17500.
Sarkar S, Reddy K, Dorgan A, Fidopiastis C, Giering M. Wearable EEG-based activity recognition in PHM-related service environment via deep learning. Int. J. Progn. Health Manag, 2016; 7: 1-10.
Längkvist M, Karlsson L, Loutfi A. Sleep stage classification using unsupervised feature learning. Advances in Artificial Neural Systems. 2012; 2012:1-9.
Zhang J, Wu Y, Bai J, Chen F. Automatic sleep stage classification based on sparse deep belief net and combination of multiple classifiers. Transactions of the Institute of Measurement and Control. 2016; 38: 435-451.
Dong H, Supratak A, Pan W, Wu C, Matthews PM, Guo Y. Mixed neural network approach for temporal sleep stage classification. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2018; 26: 324-333.
Wulsin D, Gupta J, Mani R, Blanco J, Litt B, Modeling electroencephalography waveforms with semi-supervised deep belief nets: fast classification and anomaly measurement. Journal of neural engineering. 2011; 8(3): 1-28.
Tabar YR, Halici U. A novel deep learning approach for classification of EEG motor imagery signals. Journal of neural engineering. 2016; 14:016003.
Lu N, Li T, Ren X, Miao H. A deep learning scheme for motor imagery classification based on restricted boltzmann machines. IEEE transactions on neural systems and rehabilitation engineering. 2017; 25: 566-576.
Ma T, Li H, Yang H, Lv X, Li P, Liu T, et al. The extraction of motion-onset VEP BCI features based on deep learning and compressed sensing. Journal of neuroscience methods. 2017; 275: 80-92.
Atzori M, Cognolato M, Müller H. Deep learning with convolutional neural networks applied to electromyography data: a resource for the classification of movements for prosthetic hands. Frontiers in neurorobotics. 2016; 10(9): 1:8.
Atzori M, Gijsberts A, Heynen S, Hager AGM, Deriaz O, Van Der Smagt et al. Building the Ninapro database: A resource for the biorobotics community. In: 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob). 24-27 June, Rome, Italy, 2012; p. 1258-1265.
Yan Y, Qin X, Wu Y, Zhang N, Fan J, Wang L. A restricted Boltzmann machine based two-lead electrocardiography classification. In: 12th International Conference on Wearable and Implantable Body Sensor Networks (BSN). 9-12 June, Cambridge, Massachusetts, 2015; p. 1-9.
Goldberger AL, Amaral LA, GlassL, Hausdorff JM, Ivanov PC, Mark RG et al. Physiobank, physiotoolkit, and physionet. Circulation. 2000; 101(23): e215-e220.
Ripoll VJR, Wojdel A, Romero E, Ramos P, Brugada J. ECG assessment based on neural networks with pretraining. Applied Soft Computing, 2016; 49: 399-406.
Page A, Kulkarni, A, Mohsenin T, Utilizing deep neural nets for an embedded ECG-based biometric authentication system. In: Biomedical Circuits and Systems Conference (BioCAS), 22-24 October, Atlanta, GA, USA, 2015. p. 1-4.
Ashiquzzaman A, Tushar AK, Islam MR, Shon D, Im K, Park JH et al. Reduction of Overfitting in Diabetes Prediction Using Deep Learning Neural Network. In: IT Convergence and Security 2017. Singapore: Sringer; 2018; p. 35-43.
Shashikumar SP, Shah AJ, Li Q, Clifford GD, Nemati S, A deep learning approach to monitoring and detecting atrial fibrillation using wearable technology. In: IEEE EMBS International Conference of Biomedical & Health Informatics (BHI), 4-7 March, Las Vegas, Nevada, USA, p. 141-144.
Jindal V, Birjandtalab J, Pouyan MB, Nourani M, An adaptive deep learning approach for PPG-based identification. In: 38th Annual International Conference of the Engineering in Medicine and Biology Society (EMBC), 17-20 August, Lake Buena Vista, Orlando, USA, 2016. p. 6401-6404.
Ruiz-Rodríguez JC, Ruiz-Sanmartín A, Ribas V, Caballero J, García-Roche A, Riera J et al. Innovative continuous non-invasive cuffless blood pressure monitoring based on photoplethysmography technology. Intensive care medicine. 2013; 39(9): 1618-1625.
Ordóñez FJ, Roggen D. Deep convolutional and lstm recurrent neural networks for multimodal wearable activity recognition. Sensors. 2016; 16(1): 1-25.
Saeedi R, Norgaard S, Gebremedhin AH. A closed-loop deep learning architecture for robust activity recognition using wearable sensors. In: IEEE International Conference on Big Data. 11-14 December, Boston, MA, USA, 2017; p. 473-479.
Bhattacharya S, Lane ND. From smart to deep: Robust activity recognition on smartwatches using deep learning. In: IEEE International Conference on Pervasive Computing and Communication Workshops (PerCom Workshops). 14-18 March, Sydney, Australia, 2016; pp. 1-6.
Ronao CA, Cho SB. Human activity recognition with smartphone sensors using deep learning neural networks. Expert Systems with Applications. 2016; 59: 235-244.
Ravi D, Wong C, Lo B, Yang GZ. Deep learning for human activity recognition: A resource efficient implementation on low-power devices. In: 13th International Conference on Wearable and Implantable Body Sensor Networks (BSN). 14-17 June, San Francisco, CA, USA, 2016; p. 71-76.
Eskofier BM, Lee SI, Daneault JF, Golabchi FN, Ferreira-Carvalho G, Vergara-Diaz G. et al. Recent machine learning advancements in sensor-based mobility analysis: deep learning for Parkinson's disease assessment. In: IEEE 38th Annual International Conference of the Engineering in Medicine and Biology Society (EMBC). 17-20 August, Lake Buena Vista, Orlando, USA, 2016; p. 655-658.
Ravì D, Wong C, Deligianni F, Berthelot M, Andreu-Perez J, Lo B, Yang GZ. Deep learning for health informatics. IEEE journal of biomedical and health informatics, 2017; 21(1): 4-21.
Miotto R, Wang F, Wang S, Jiang X, Dudley JT. Deep learning for healthcare: review, opportunities and challenges. Briefings in bioinformatics. 2017; 1-11.
Faust O, Hagiwara Y, Hong TJ, Lih OS, Acharya UR. Deep learning for healthcare applications based on physiological signals: a review. Computer methods and programs in biomedicine. 2018; 1-31.
Lane ND, Georgiev P. Can deep learning revolutionize mobile sensing? In: Proceedings of the 16th International Workshop on Mobile Computing Systems and Applications. 12-13 February, Santa Fe, NM, USA, 2015; p. 117-122.
Yao S, Hu S, Zhao Y, Zhang A, Abdelzaher T. Deepsense: A unified deep learning framework for time-series mobile sensing data processing. In: Proceedings of the 26th International Conference on World Wide Web. 3-7 April, Perth, Australia, 2017; p. 351-360.
Lane ND, Bhattacharya S, Georgiev P, Forlivesi C, Jiao L, Qendro L, Kawsar F. Deepx: A software accelerator for low-power deep learning inference on mobile devices. In: 15th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN). 11-14 April, Vienna, Austria, 2016; p. 1-12.