Electrogastrogram-based detection of cybersickness with the application of wavelet transformation and machine learning: a case study
Abstract
Introduction/purpose: The application of virtual reality (VR) and simulation technologies in military training offers cost-effective and versatile approach to training enhancement. However, prevalence of cybersickness (CS), characterized by symptoms such as nausea, limits their widespread use.
Methods: This study introduces objective parameters for the detection of CS using three-channel electrogastrogram (EGG) recording from one specific subject and assesses the independence and linear correlation for appropriate channel selection. The paper employs a 3-level discrete wavelet transformation (DWT) on the chosen channel to identify key parameters indicative of gastric disturbances. Furthermore, the paper investigates recovery from CS following VR and examines the application of unsupervised machine learning (ML) for segmenting EGG into baseline and CS, utilizing significant features previously identified.
Results and discussion: The analysis reveals no significant differences across EGG channels and moderate to low linear correlation between channel pairs. The feature selection demonstrates that the root mean square of the amplitude as well as the maximum and mean values of the power spectral density (PSD) calculated on all DWT coefficients, are effective for CS detection while the dominant EGG scale could not indicate CS for any level of decomposition. Furthermore, recovery signs appear approximately 8 minutes after the first VR experience supporting the idea of conducting multiple sessions the same day \textit{i.e.}, intensive VR-based training.
Conclusions: The unsupervised ML shows potential in identifying CS-affected EGG signal segments with feature extraction based on DWT, offering a novel approach for enhancing the prevention of CS occurrence in VR-based military training and other VR-related environments.
References
Akansu, A.N. & Haddad, R.A. 2001. Multiresolution signal decomposition: Transforms, subbands, and wavelets, 2nd Edition. San Diego: Academic Press. ISBN 978-0-12-047141-6.
Bruzzone, A.G. & Massei, M. 2017. Simulation-Based Military Training. In: Mittal, S., Durak, U. & Ören, T. (Eds.) Guide to Simulation-Based Disciplines. Simulation Foundations, Methods and Applications. pp. 315–361. Cham: Springer. Available at: https://doi.org/10.1007/978-3-319-61264-5_14.
Cacioppo, J.T., Tassinary, L.G. & Berntson, G. 2007. Handbook of psychophysiology. Cambridge University Press. ISBN 9781139461931.
Dennison, M.S., D’Zmura, M., Harrison, A.V., Lee, M. & Raglin, A.J. 2019. Improving motion sickness severity classification through multi-modal data fusion. In: Pham, T. (Eds.) Proceedings Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications. Vol. 11006, art.number:110060T. Available at: https://doi.org/10.1117/12.2519085.
Dennison, M.S., Zachary Wisti, A. & D’Zmura, M. 2016. Use of physiological signals to predict cybersickness. Displays, 44, pp. 42–52. Available at: https://doi.org/10.1016/j.displa.2016.07.002.
Gruden, T., Popović, N.B., Stojmenova, K., Jakus, G., Miljković, N., Tomažič, S. & Sodnik, J. 2021. Electrogastrography in Autonomous Vehicles—An Objective Method for Assessment of Motion Sickness in Simulated Driving Environments. Sensors, 21(2), art.number:550. Available at: https://doi.org/10.3390/s21020550.
Géron, A. 2022. Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow (3rd ed). Sebastopol, California: O’Reilly Media, Inc. ISBN 9781098122478.
Harris, C.R., Millman, K.J., Van Der Walt, S.J., Gommers, R., Virtanen, P., Cournapeau, D., Wieser, E., Taylor, J., Berg, S., Smith, N.J., Kern, R., Picus, M., Hoyer, S., Van Kerkwijk, M.H., Brett, M., Haldane, A., Del Río, J.F., Wiebe, M., Peterson, P., Gérard-Marchant, P., Sheppard, K., Reddy, T., Weckesser, W., Abbasi, H., Gohlke, C. & Oliphant, T.E. 2020. Array programming with NumPy. Nature, 585, pp. 357–362. Available at: https://doi.org/10.1038/s41586-020-2649-2.
Hunter, J.D. 2007. Matplotlib: A 2D Graphics Environment. Computing in Science & Engineering, 9(3), pp. 90–95. Available at: https://doi.org/10.1109/MCSE.2007.55.
Isu, N., Hasegawa, T., Takeuchi, I. & Morimoto, A. 2014. Quantitative analysis of time-course development of motion sickness caused by in-vehicle video watching. Displays, 35(2), pp. 90–97. Available at: https://doi.org/10.1016/j.displa.2014.01.003.
Jakus, G., Sodnik, J. & Miljković, N. 2022. Electrogastrogram-Derived Features for Automated Sickness Detection in Driving Simulator. Sensors, 22(22), art.number:8616. Available at: https://doi.org/10.3390/s22228616.
Kaiser, J. & Schafer, R. 1980. On the use of the I0-sinh window for spectrum analysis. IEEE Transactions on Acoustics, Speech, and Signal Processing, 28(1), pp. 105–107. Available at: https://doi.org/10.1109/TASSP.1980.1163349.
Kara, S., Dirgenali, F. & Okkesim, S. 2005. Estimating Gastric Rhythm Differences Using a Wavelet Method from the Electrogastrograms of Normal and Diabetic Subjects. Instrumentation Science & Technology, 33(5), pp. 519–532. Available at: https://doi.org/10.1080/10739140500222907.
Keshavarz, B. & Hecht, H. 2011. Validating an Efficient Method to Quantify Motion Sickness. Human Factors: The Journal of the Human Factors and Ergonomics Society, 53(4), pp. 415–426. Available at: https://doi.org/10.1177/0018720811403736.
Keshavarz, B., Peck, K., Rezaei, S. & Taati, B. 2022. Detecting and predicting visually induced motion sickness with physiological measures in combination with machine learning techniques. International Journal of Psychophysiology, 176, pp. 14–26. Available at: https://doi.org/10.1016/j.ijpsycho.2022.03.006.
Kim, Y.Y., Kim, H.J., Kim, E.N., Ko, H.D. & Kim, H.T. 2005. Characteristic changes in the physiological components of cybersickness. Psychophysiology, 42(5), pp. 616–625. Available at: https://doi.org/10.1111/j.1469-8986.2005.00349.x.
Kuhl, J., Evans, D., Papelis, Y., Romano, R. & Watson, G. 1995. The Iowa Driving Simulator: an immersive research environment. Computer, 28(7), pp. 35–41. Available at: https://doi.org/10.1109/2.391039.
LaViola, J.J. 2000. A discussion of cybersickness in virtual environments. ACM SIGCHI Bulletin, 32(1), pp. 47–56. Available at: https://doi.org/10.1145/333329.333344.
Lee, B.H. & Kuo, S. 2001. Real-Time Digital Signal Processing: Implementaions, Applications and Experiements with the TMS320C55X. Wiley Hoboken, NJ, USA. ISBN 1-280-55452-5.
McKinney, W. 2010. Data Structures for Statistical Computing in Python. In: Van der Walt, S. & Millman, J. (Eds.) Proceedings of the 9th Python in Science Conference. Austin, Texas, pp.56-61, June 28 - July 3. Available at: https://doi.org/10.25080/Majora-92bf1922-00a.
McKnight, P.E. & Najab, J. 2010. Mann-Whitney U Test. The Corsini Encyclopedia of Psychology. 30 January. Available at: https://doi.org/10.1002/9780470479216.corpsy0524.
Merchant, W. & Kirollos, R. 2022. An Overview of Cybersickness Self-Report Measures for use in Defence Research and Development Canada Experiments, Reference Document DRDC-RDDC-2022-D063. Defence Research and Development Canada. [online]. Available at: https://cradpdf.drdc-rddc.gc.ca/PDFS/unc392/p814963_A1b.pdf [Accessed: 10 June 2024].
Meta. 2024. Epic Roller Coasters [online]. Available at: https://www.meta.com/experiences/pcvr/epic-roller-oasters/1477883658957255/#reviews [Accessed: 10 June 2024].
Miljković, N., Popović, N.B., Prodanov, M. & Sodnik, J. 2019. Assessment of sickness in virtual environments. In: Konjović, Z., Zdravković, M. & Trajanović, M. (Eds.) Proceedings of the 9th International Conference on Information Society and Technology (ICIST 2019). Kopaonik, Serbia, pp.76-81, March 10-13 [online]. Available at: https://www.eventiotic.com/eventiotic/files/Papers/URL/f412a358-82d8-46aa-b0c9-6ad30de5890d.pdf [Accessed: 10 June 2024] ISBN: 978-86-85525-24-7.
Miljković, N., Popović, N.B. & Sodnik, J. 2023. Electrogastrogram Signal Processing: Techniques and Challenges with Application for Simulator Sickness Assessment. In: Biomedical Signal Processing, 1st Edition, pp.62-89. CRC Press [online]. Available at: https://www.taylorfrancis.com/chapters/edit/10.1201/9781003201137-4/electrogastrogram-signal-processing-nadica-miljkovi%C4%87-nenad-popovi%C4%87-jaka-sodnik [Accessed: 10 June 2024] ISBN: 9781003201137.
Miljković, N. & Sodnik, J. 2023. Towards Objective Assessment of Driving Simulation Sickness: Pros and Cons of Stomach Electrical Activity. In: Kemeny, A., Chardonnet, J.-R. & Colombet, F. (Eds.) Proceedings of the Driving Simulation Conference 2023 Europe VR. Antibes Juan-les-Pins, France, pp.81-88, September 6-8 [online]. Available at: https://proceedings.driving-simulation.org/proceeding/dsc-2023/towards-objective-assessment-of-driving-simulation-sickness -pros-and-cons-of-stomach-electrical-activity/ [Accessed: 10 June 2024] ISBN: 978-2-9573777-3-2.
NATO Science and Technology Office. 2021. Guidelines for Mitigating Cybersickness in Virtual Reality Systems. Peer-reviewed Final Report of the Human Factors and Medicine Panel/Modeling and Simulations Group, Activity Number 323 (NATO STO-TR-HFM-MSG-323) [online]. Available at: https://apps.dtic.mil/sti/pdfs/AD1183673.pdf [Accessed: 10 June 2024].
O’Grady, G., Angeli, T.R., Paskaranandavadivel, N., Erickson, J.C., Wells, C.I., Gharibans, A.A., Cheng, L.K. & Du, P. 2018. Methods for High-Resolution Electrical Mapping in the Gastrointestinal Tract. IEEE Reviews in Biomedical Engineering, 12, pp. 287–302. Available at: https://doi.org/10.1109/RBME.2018.2867555.
O’Grady, G., Du, P., Cheng, L.K., Egbuji, J.U., Lammers, W.J.E.P., Windsor, J.A. & Pullan, A.J. 2010. Origin and propagation of human gastric slow-wave activity defined by high-resolution mapping. American Journal of Physiology-Gastrointestinal and Liver Physiology, 299(3), pp. G585–G592. Available at: https://doi.org/10.1152/ajpgi.00125.2010.
Oppenheim, A.V. 2008. Discrete-time Signal Processing, 2nd Edition. Pearson Education. ISBN 9788131704929.
Parkman, H.P., Hasler, W.L., Barnett, J. & Eaker, E. 2003. Electrogastrography: a document prepared by the gastric section of the American Motility Society Clinical GI Motility Testing Task Force. Neurogastroenterology & Motility, 15(2), pp. 89–102. Available at: https://doi.org/10.1046/j.1365-2982.2003.00396.x.
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M. & Duchesnay, E. 2011. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research, 12, pp. 2825–2830 [online]. Available at: https://www.jmlr.org/papers/volume12/pedregosa11a/pedregosa11a.pdf [Accessed: 10 June 2024].
Percival, D.B. & Walden, A.T. 2006. Wavelet Methods for Time Series Analysis (Cambridge Series in Statistical and Probabilistic Mathematics, Series Number 4). Cambridge University Press. ISBN 9780521685085.
Popović, N.B. 2021. Methods for assessment of electrical activity of smooth muscles. Phd thesis, Belgrade, Serbia: University of Belgrade, School of Electrical Engineering. [online]. Available at: https://nardus.mpn.gov.rs/handle/123456789/18615 [Accessed: 10 June 2024].
Popović, N.B., Miljković, N. & Popović, M.B. 2019a. Simple gastric motility assessment method with a single-channel electrogastrogram. Biomedical Engineering/Biomedizinische Technik, 64(2), pp. 177–185. Available at: https://doi.org/10.1515/bmt-2017-0218.
Popović, N.B., Miljković, N., Stojmenova, K., Jakus, G., Prodanov, M. & Sodnik, J. 2019b. Lessons Learned: Gastric Motility Assessment During Driving Simulation. Sensors, 19(14), art.number:3175. Available at: https://doi.org/10.3390/s19143175.
Seabold, S. & Perktold, J. 2010. Statsmodels: Econometric and Statistical Modeling with Python. In: Van der Walt, S. & Millman, J. (Eds.) Proceedings of the 9th Python in Science Conference. Austin, Texas, pp.92-96, June 28-July 3. Available at: https://doi.org/10.25080/Majora-92bf1922-011.
Stern, R.M., Koch, K.L., Stewart, W.R. & Lindblad, I.M. 1987. Spectral analysis of tachygastria recorded during motion sickness. Gastroenterology, 92(1), pp. 92–97. Available at: https://doi.org/10.1016/0016-5085(87)90843-2.
Tecknotrove. 2023. Role of defense simulator training in enhancing national security. Tecknotrove.com, 26 September [online]. Available at: https://tecknotrove.com/role-of-defence-simulator-training-in-national-security/ [Accessed: 10 June 2024].
Tian, N., Achache, K.H., Ben Mustapha, A.R. & Boulic, R. 2023. EGG Objective characterization of cybersickness symptoms towards navigation axis. In: 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). Shanghai, China, pp.289-297, March 25-28. Available at: https://doi.org/10.1109/VRW58643.2023.00068.
Tian, N. & Boulic, R. 2024. Who says you are so sick? An investigation on individual susceptibility to cybersickness triggers using EEG, EGG and ECG. IEEE Transactions on Visualization and Computer Graphics, 30(5), pp. 2379–2389. Available at: https://doi.org/10.1109/TVCG.2024.3372066.
Tokmakçi, M. 2007. Analysis of the Electrogastrogram Using Discrete Wavelet Transform and Statistical Methods to Detect Gastric Dysrhythmia. Journal of Medical Systems, 31(4), pp. 295–302. Available at: https://doi.org/10.1007/s10916-007-9069-9.
Valens, C. 1999. A really friendly guide to wavelets. [online]. Available at: https://www.cs.unm.edu/~williams/cs530/arfgtw.pdf [Accessed: 10 June 2024].
Van Rossum, G. & Drake, F.L. 1995. Python reference manual, Release 2.1.1. Amsterdam, The Netherlands: Stichting Mathematisch Centrum. ISBN 1441412697.
Virtanen, P., Gommers, R., Oliphant, T.E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., van der Walt, S.J., Brett, M., Wilson, J., Millman, K.J., Mayorov, N., Nelson, A.R.J., Jones, E., Kern, R., Larson, E., Carey, C.J., Polat, I., Feng, Y., Moore, E.W., VanderPlas, J. & SciPy1.0Contributors. 2020. SciPy 1.0: fundamental algorithms for scientific computing in Python. Nature methods, 17, pp. 261–272. Available at: https://doi.org/10.1038/s41592-019-0686-2.
Vlačić, S.I., Knežević, A.Z., Mandal, S., Rođenkov, S. & Vitsas, P. 2020. Improving the pilot selection process by using eye-tracking tools. Journal of Eye Movement Research, 12(3). Available at: https://doi.org/10.16910/jemr.12.3.4.
Woolson, R.F. 2005. Wilcoxon Signed-Rank Test. In: Encyclopedia of Biostatistics. 15 July. Available at: https://doi.org/10.1002/0470011815.b2a15177.
Yang, A.H.X., Kasabov, N. & Cakmak, Y.O. 2022. Machine learning methods for the study of cybersickness: a systematic review. Brain Informatics, 9, art.number:24. Available at: https://doi.org/10.1186/s40708-022-00172-6.
Copyright (c) 2025 Ilija V. Tanasković, Nenad B. Popović, Jaka J. Sodnik, Sašo J. Tomažič, Nadica S. Miljković
This work is licensed under a Creative Commons Attribution 4.0 International License.
Proposed Creative Commons Copyright Notices
Proposed Policy for Military Technical Courier (Journals That Offer Open Access)
Authors who publish with this journal agree to the following terms:
Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).