- Research
- Open access
- Published:
Deep learning model for classifying shoulder pain rehabilitation exercises using IMU sensor
Journal of NeuroEngineering and Rehabilitation volume 21, Article number: 42 (2024)
Abstract
Background
Artificial intelligence is being used for rehabilitation, including monitoring exercise compliance through sensor technology. AI classification of shoulder exercise wearing an IMU sensor has only been reported in normal (i.e. painless) subjects. To prove the feasibility of monitoring exercise compliance, we aimed to classify 11 types of shoulder rehabilitation exercises using an AI (artificial intelligence) algorithm in patients with shoulder pain. We had the patients wear an IMU-based sensor, collected data during exercise, and determined the accuracy of exercise classification.
Methods
Data were collected from 58 patients (27 males, 31 females, age range 37–82 years) diagnosed with shoulder diseases such as adhesive capsulitis and rotator cuff disease. 11 types of shoulder pain rehabilitation exercise programs were developed and repeated each exercise ten times per session while wearing an IMU sensor. The study applied the Rectified Linear Unit (ReLU) and the SoftMax as the activation function for hidden layers, the output layer.
Results
The acquired data was used to train a DNN model using the multilayer perceptron algorithm. The trained model was used to classify 11 types of shoulder pain rehabilitation exercises. The training accuracy was 0.975 and the test accuracy was 0.925.
Conclusion
The study demonstrates that IMU sensor data can effectively classify shoulder pain rehabilitation exercises, providing more appropriate feedback for patients. The model can be utilized to establish a system for remotely monitoring patients’ exercise performance. The use of deep learning in patient monitoring and rehabilitation has significant potential to bring innovative changes to healthcare service delivery.
Introduction
Shoulder joint disorders are estimated to account for 16% of the prevalence of musculoskeletal disorders and can cause pain, decreased muscle strength, numbness, paresthesia, and fatigue [1]. For most shoulder pain patients, conservative treatments such as nonsteroidal anti-inflammatory drugs (NSAIDs), joint injections, activity and work modification, physical therapy, manual therapy, botulinum toxin, and surgical intervention are generally considered first [2, 3]. After reducing pain through medication and injections, it is important to restore shoulder function through exercise therapy.
Exercise therapy is essential to normalize shoulder biomechanics and joint range of motion and can prevent disease recurrence [4,5,6]. To ensure exercise therapy is effective, compliance is crucial. Although medical professionals recommend that patients perform home-based exercises, patients often have difficulty performing home-based exercises consistently and independently, including the fact that home-based exercises may be limited without sufficient feedback or guidance. Previous research shows that up to 65% of patients do not adhere to home-based exercise programs, and approximately 10% fail to complete prescribed exercise therapy courses [7]. Without proper guidance and feedback, individuals may perform exercises incorrectly, risking injury or potential hazards. Recent advancements in technology have significantly impacted medical fields, offering new possibilities for objective monitoring and data recording in human movement [8,9,10].
Inertial Measurement Unit (IMU) sensors, combining accelerometers, gyroscopes, and magnetometers, have emerged as promising tools for capturing movement data during rehabilitation exercises. These sensors can accurately measure movements, directions, and positions in 3D space, providing detailed information on exercise performance [11,12,13,14,15]. With the use of IMU sensors, it is possible to monitor the movement of the patients during exercise.
Monitoring the movement during shoulder exercise therapy using an IMU sensor has been reported [16, 17]. However, the subjects in previous studies were healthy adults. To the knowledge of the authors, there are no previous studies in patients with disorders accompanied by shoulder pain. Patients, depending on their conditions and health status, require customized exercises, which limits the opportunity to utilize automatic exercise classification technologies.
The aim of this study is to train and test a Deep Neural Network (DNN) model to automatically classify 11 types of shoulder pain rehabilitation exercises using data collected from patients with shoulder pain. Using IMU sensor data, which captures three-axis accelerometer and gyroscope measurements of the shoulder joint during exercise, the DNN model is designed to identify different movement patterns associated with specific exercises.
Methods
Data acquisition
IMU sensor data was collected from participants in rehabilitation exercises for shoulder pain, The participants diagnosed with shoulder diseases such as adhesive capsulitis and rotator cuff disease were recruited from SMG-SNU Boramae Medical Center. The wearable sensor used for data collection (31 mm × 61 mm × 15 mm) was developed and used internally by the research team. It was designed to be worn like a wristwatch, can be fitted with a strap, and is equipped with an IMU sensor. An integrated sensor (MPU9250; InvenSense, San Jose, CA, USA) comprising a gyroscope (range, 2,000°/s), an accelerometer (range, 16 g), and a magnetometer (range, 49 G) was implemented as the inertial sensor for the module. The microcontroller was programmed to receive signals from the sensor, at a frequency of 100 Hz, through the SPI connection. The obtained angular velocity, acceleration, and magnetometer data were merged, and wireless Bluetooth module (PAN1321i; Panasonic, Osaka, Japan) was used to transmit the calculated data to a computer (Fig. 1).
Through a visual program, the accelerometer and gyroscope of the IMU sensor can be monitored in real-time. The sensors were calibrated before data collection to ensure accurate measurements. The patients visited the hospital for rehabilitation exercise a minimum of 1 time and a maximum of 9 times, with 10 repetitions of the same exercise performed during each visit. To ensure accuracy and comprehension, 11 types of shoulder rehabilitation exercises were provided to patients via instructional videos (Table 1), and throughout the data collection process, the patients were monitored by clinicians or researchers to ensure proper exercise performance and reduce the risk of injury (Fig. 2).
Data preprocessing
The collected IMU sensor data was preprocessed to prepare it for training the deep neural network model. This preprocessing included several steps We identified the start and end times of exercise sessions to separate exercise data from rest periods, enabling our AI system to concentrate on pertinent and substantial data. Any erroneous or noisy data points were removed from the dataset by applying filtering program to identify and handle any inconsistencies in the sensor measurements.
Feature extraction
Twenty-four distinct features were obtained by extracting four statistical indicators (minimum, maximum, average, and variance) from the x, y, and z axes of both acceleration and angular velocity.
Data split
The dataset were divided into training, and testing sets. The training set was used to train the deep neural network model, the testing set was used to evaluate the final performance of the model.
Deep neural network model (DNN)
Deep Neural Network (DNN) is a potent tool that can closely determine elements impacting 3D positioning, as well as learn complex data patterns [18]. There has been recent interest in using multi-layer structured DNN to learn IMU sensor data, particularly in the fields of classification and numerical prediction [8, 9]. The DNN was constructed to classify the shoulder pain rehabilitation exercises based on the preprocessed IMU sensor data. The model included an input layer for the 24 features, and five hidden layers, each having 8, 16, 32, 64, 128 nodes, respectively, which aid the model in learning complex patterns and relationships in the data. The Rectified Linear Unit (ReLU) was used as the activation function for hidden layers, and the Softmax was used for the output layer [19, 20] (Fig. 3).
-
1.
Input Layer: The input layer receives the preprocessed IMU sensor data, which acts as the model’s features.
-
2.
Hidden Layers: five hidden layers are added to the model to capture the complex relationships and patterns in the data.
-
3.
Output Layer: The output layer produces the classification predictions for the different shoulder pain rehabilitation exercises.
-
4.
Activation Functions: The Rectified Linear Unit (ReLU) activation function is utilized in all hidden layers, facilitating efficient training by addressing the vanishing gradient problem and promoting faster convergence [19]. The Softmax activation function is employed in the output layer, converting the network’s output into a probability distribution over predicted output classes, and thus is suitable for multi-class classification tasks [20].
Model training and evaluation
The preprocessed data was loaded and then split into training and test sets at an 8:2 ratio. Subsequently, a DNN model was trained by inputting the training data into the model. The preprocessed acceleration and angular velocity data from the test set were then input into the trained DNN model to predict the type of 11 shoulder rehabilitation exercises. The predicted workouts were compared to the patients’ actual data, and their accuracy was calculated using the accuracy score function from the scikit-learn library.
Results
General characteristics of participants
Fifty-eight participants diagnosed with shoulder disease participated in this study. Of the participants, 27 were male and 31 were female. The age range of the participants was 37 to 82 years, with the most prevalent age group being those aged 60 to under 70(53.4%). The average age of the participants was 60.5 years. To assess the severity of shoulder pain and disability, we used The Shoulder Pain and Disability Index(SPADI) questionnaire. It consists of 13 items that evaluate both pain and the impact of pain on various daily activities. The SPADI provides a total score ranging from 0 to 100, with higher scores indicating more severe pain and disability [21, 22](Table 2).
Classification accuracy
When evaluating the performance of the developed deep learning model, two primary metrics were considered: train accuracy and test accuracy. The train accuracy reached 0.975, indicating the model’s proficiency in correctly classifying the shoulder rehabilitation exercises during the training phase. This high level of accuracy suggests that the model has effectively learned the patterns and distinguishing features from the training dataset, which consisted of 80% of the collected data. The model’s test accuracy of 0.925 suggests that it maintained high classification performance when exposed to the remaining 20% of the data, which it had not encountered during the training phase. The proximity of the test accuracy to the train accuracy indicates the model’s robustness and potential applicability in real-world scenarios, where maintaining accuracy on data beyond the training set is critical for the algorithm.
In this study, a heatmap was used for visualizing the model’s performance. The heatmap displays the level of agreement between actual values and predictive values, with the intensity of color indicating the degree of agreement. The diagonal direction of the heatmap is represented by dark colors, indicating that the model has made accurate predictions [10, 23]. This suggests that the model is highly accurate and reliable (Fig. 4). Out of the 11 exercises, the one with the highest prediction accuracy was ‘5) Shoulder abduction (with band)’ while the one with the lowest prediction accuracy was ‘4) Lawnmower exercise’. Notably, exercises ‘6) Shoulder extension (with band)’ and ‘7) Shoulder external rotation (with band)’ are also depicted, highlighting an interesting observation from the heatmap. The model exhibits cross-identification, predicting exercise 6 as 7 and vice versa. This misclassification between the two exercise types may suggest similar feature representations within the model’s learned parameters.
Discussion
The research results presented in this paper show that 11 shoulder rehabilitation exercise programs performed with one wearable IMU sensor can be distinguished with 92.5% accuracy. The findings of this study are in line with previous research in Human Activity Recognition (HAR) that utilized IMU sensors to analyze human movements and distinguish them through deep learning models. Moreover, the significance of this research lies in its basis on actual patient data. The study focuses on patients with shoulder pain, rather than on routine exercises for healthy individuals. This provides valuable data that can be applied in future research on rehabilitation exercises for a broader range of diseases.
As the data were collected from patients with varying severity of shoulder pain, we expected that the pattern would be more complex and difficult to predict than the movements of the healthy general population. However, a data collection program was developed to mark exercises’ start and end points with specific numbers, making the data pre-processing much easier. As a result, it was possible to distinguish exercises with relatively high accuracy for patients.
Furthermore, as observed in the heatmap results, exercise ‘5) shoulder abduction (with band)’ demonstrated the highest prediction rate compared to other exercises. This is presumably because the ‘5) shoulder abduction (with band)’ movement occurs within a single frontal plane and involves a wider range of motion, resulting in relatively uncomplicated IMU sensor data that is easier for the model to learn. Conversely, the exercise with the lowest prediction rate, ‘4) Lawnmower exercise’, involves movements across all three planes and allows for adjustment of motion range based on the intensity of the patient’s pain, which could have led to the collection of complex and diverse data (Fig. 5).
The Heatmap results of the classification experiment confirmed that exercise ‘6) Shoulder extension (with band)’ and ‘7) Shoulder external rotation (with band)’ were confused with each other with a probability of 0.018 to 0.02. This suggests that the data collected during the two exercises share similar features. Comparing the sensor data from the two exercises, we found that the sensor, when worn on the wrist, showed movements at remarkably similar positions, with comparable velocities, and along nearly identical trajectories (Fig. 6). An IMU sensor can track an object’s movement or estimate its posture by measuring its position, direction, and speed in three-dimensional space. Therefore, distinguishing between two different movements using only an IMU sensor can be difficult if the sensor’s position, direction, and speed values are very similar, even though they represent two different movements.
To identify the underlying factors contributing to these results, further research is necessary, taking into consideration variables such as exercise difficulty, range of motion, and duration of movements, to assess their impact on classification accuracy.
In hospitals, exercise prescriptions are given for rehabilitation treatment of musculoskeletal disorders, and patients can perform exercises under the direction of a physical or occupational therapist. However, once patients return home, it can be difficult for patients to continue exercising on their own and receive professional feedback when they return home. The study confirms that a single wristwatch-type sensor can automatically classify the type of exercise performed using a deep learning model. If this deep learning model is utilized to develop a home training service, combined with a mobile application or TV, patients will be able to exercise at home while wearing sensors. Medical professionals will have access to the patient’s exercise records. Based on this data, they can provide feedback and motivation to help patients continue exercising. The recorded information can also be used for future research on patients.
Consistent performance of rehabilitation therapy is extremely important. To prevent the chronicity of diseases and facilitate a return to daily life, devices that assist in performing accurate and regular movements are essential. Therefore, such technology is actively researched in the field of rehabilitation medicine and holds significant value for future investigation. The ability to monitor exercises could also be applied to telemedicine services like remote exercise prescriptions, which could result in positive social and economic effects.
However, this study has limitations. Because the DNN model was trained and validated using an 80:20 split within the same dataset, it necessitates validation against data collected in new environments. Additionally, while the amount of data used for training was not small, the fact that the subject group was composed only of patients with shoulder joint diseases experiencing shoulder pain limits the study by not utilizing data for various shoulder conditions. Since the treatment applied varies according to the specific diagnosis of shoulder disorders, there is a need for future research to categorize exercise programs more finely according to precise shoulder diagnoses.
Conclusion
In conclusion, the application of deep neural networks to the classification of shoulder pain rehabilitation exercises using IMU sensor data shows promise for improving the monitoring and assessment of exercise performance. This technology has the potential to benefit both clinicians and individuals by optimizing exercise performance and improving outcomes in shoulder pain rehabilitation programs. Further research is necessary to enhance the model’s validity and applicability, firstly by expanding the dataset to include a wider range of shoulder disease severity and rehabilitation exercises for each severity. Additionally, investigating the model’s applicability in clinical settings, particularly in remote monitoring and patient adherence to prescribed exercises, is a vital next step. Our objective is to contribute to the development of an intelligent and personalized rehabilitation platform that can adapt to the unique needs of each patient. This will optimize recovery outcomes and enhance the quality of care.
Data availability
No datasets were generated or analysed during the current study.
References
Burbank KM, et al. Chronic shoulder pain: part I. evaluation and diagnosis. Am Family Phys. 2008;77(4):453–60.
Greenberg DL. Evaluation, and treatment of shoulder pain. Med Clin. 2014;98(3):487–504.
Garving C, et al. Impingement syndrome of the shoulder. Deutsches Ärzteblatt International. 2017;114(45):765.
Marinko LN, et al. The effectiveness of therapeutic exercise for painful shoulder conditions: a meta-analysis. J Shoulder Elbow Surg. 2011;20(8):1351–9.
Desmeules F, Côté CH, Frémont P. Therapeutic exercise and orthopedic manual therapy for impingement syndrome: a systematic review. Clin J Sports Med. 2003;13(3):176–82.
Geraets JJ, et al. Effectiveness of a graded exercise therapy program for patients with chronic shoulder complaints. Australian J Physiotherapy. 2005;51(2):87–94.
Bassett SF. The assessment of patient adherence to physiotherapy rehabilitation. NZ J Physiotherapy. 2003;31(2):60–6.
Chen K, et al. Deep learning for sensor-based human activity recognition: overview, challenges, and opportunities. ACM Comput Surv (CSUR). 2021;54(4):1–40.
Chen C. Deep Learning for Inertial Positioning: A Survey arXiv preprint arXiv:2303.03757, 2023.
O’Reilly MA, et al. Classification of lunge biomechanics with multiple and individual inertial measurement units. Sports Biomech. 2017;16(3):342–60.
Giggins O, Kelly D, Caulfield B. Evaluating rehabilitation exercise performance using a single inertial measurement unit. in 2013 7th International Conference on Pervasive Computing Technologies for Healthcare and Workshops. 2013. IEEE.
Whelan DF, et al. Technology in rehabilitation: evaluating the single leg squat exercise with wearable inertial measurement units. Methods Inf Med. 2017;56(02):88–94.
Leardini A, et al. Validation of the angular measurements of a new inertial-measurement-unit based rehabilitation system: comparison with state-of-the-art gait analysis. J Neuroeng Rehabil. 2014;11(1):1–7.
Zhuang W, et al. Design of human activity recognition algorithms based on a single wearable IMU sensor. Int J Sens Networks. 2019;30(3):193–206.
Dehzangi O, Sahu V. IMU-based robust human activity recognition using feature analysis, extraction, and reduction. in 2018 24th International Conference on Pattern Recognition (ICPR). 2018. IEEE.
Panwar M, et al. Rehab-net: deep learning framework for arm movement classification using wearable sensors for stroke rehabilitation. IEEE Trans Biomed Eng. 2019;66(11):3026–37.
Taylor PE et al. Classifying human motion quality for knee osteoarthritis using accelerometers. in. 2010 Annual International Conference of the IEEE Engineering in medicine and Biology. 2010. IEEE.
Sansano E, Montoliu R, Belmonte Fernandez O. A study of deep neural networks for human activity recognition. Comput Intell. 2020;36(3):1113–39.
Szandała T. Review and comparison of commonly used activation functions for deep neural networks Bio-inspired neurocomputing, 2021: p. 203–224.
Sharma S, Sharma S, Athaiya A. Activation functions in neural networks. Towards Data Sci. 2017;6(12):310–6.
Roach KE, et al. Development of a shoulder pain and disability index. Arthritis Rheumatism: Official J Am Coll Rheumatol. 1991;4(4):143–9.
González Aroca J, et al. Fear-avoidance beliefs are Associated with Pain Intensity and Shoulder disability in adults with Chronic Shoulder Pain: a cross-sectional study. J Clin Med. 2023;12(10):3376.
Konak O, Wegner P, Arnrich B. IMU-Based Movement Trajectory Heatmaps for Human Activity Recognition. Sensors. 2020;20(24):7179.
Funding
This research was supported by the Culture, Sports and Tourism R&D Program through the Korea Creative Content Agency (KOCCA) grant funded by the Ministry of Culture, Sports and Tourism (MCST) in 2023(Project Name: Development of infrastructure linkage technology for community-based rehabilitation movements, Project Number: R202106001, Contribution Rate: 100%).
Author information
Authors and Affiliations
Contributions
K.L: Conceptualization, Methodology, Software, Writing - Original Draft Preparation, Visualization, Investigation.S.L (Corresponding Author): Supervision, Project Administration, Writing - Review & Editing, Funding Acquisition. J.K. : Writing - Review & Editing (Introduction and Discussion sections), Contributing Ideas. H.H. : Participant Management, Data Collection .Y.J. : Data Management, Data Verification and Validation. H.R. and H.K. : Data Preprocessing, Deep Learning Model Training, Model Testing, Result Interpretation, and Analysis.All authors have reviewed the manuscript and contributed significantly to its completion. S.L., as the corresponding author, provided overall guidance for the research project and manuscript preparation process.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
The study received approval from the Seoul Metropolitan Government Boramae Medical Center institutional review board (Approval No.: 20-2022-1), and all participants provided written informed consent.
Consent for publication
All participants provided written and informed consent for the publication of these data.
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Lee, K., Kim, JH., Hong, H. et al. Deep learning model for classifying shoulder pain rehabilitation exercises using IMU sensor. J NeuroEngineering Rehabil 21, 42 (2024). https://doi.org/10.1186/s12984-024-01343-8
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s12984-024-01343-8