Skip to main content

A systematic review of the applications of markerless motion capture (MMC) technology for clinical measurement in rehabilitation

Abstract

Background

Markerless motion capture (MMC) technology has been developed to avoid the need for body marker placement during motion tracking and analysis of human movement. Although researchers have long proposed the use of MMC technology in clinical measurement—identification and measurement of movement kinematics in a clinical population, its actual application is still in its preliminary stages. The benefits of MMC technology are also inconclusive with regard to its use in assessing patients’ conditions. In this review we put a minor focus on the method’s engineering components and sought primarily to determine the current application of MMC as a clinical measurement tool in rehabilitation.

Methods

A systematic computerized literature search was conducted in PubMed, Medline, CINAHL, CENTRAL, EMBASE, and IEEE. The search keywords used in each database were “Markerless Motion Capture OR Motion Capture OR Motion Capture Technology OR Markerless Motion Capture Technology OR Computer Vision OR Video-based OR Pose Estimation AND Assessment OR Clinical Assessment OR Clinical Measurement OR Assess.” Only peer-reviewed articles that applied MMC technology for clinical measurement were included. The last search took place on March 6, 2023. Details regarding the application of MMC technology for different types of patients and body parts, as well as the assessment results, were summarized.

Results

A total of 65 studies were included. The MMC systems used for measurement were most frequently used to identify symptoms or to detect differences in movement patterns between disease populations and their healthy counterparts. Patients with Parkinson’s disease (PD) who demonstrated obvious and well-defined physical signs were the largest patient group to which MMC assessment had been applied. Microsoft Kinect was the most frequently used MMC system, although there was a recent trend of motion analysis using video captured with a smartphone camera.

Conclusions

This review explored the current uses of MMC technology for clinical measurement. MMC technology has the potential to be used as an assessment tool as well as to assist in the detection and identification of symptoms, which might further contribute to the use of an artificial intelligence method for early screening for diseases. Further studies are warranted to develop and integrate MMC system in a platform that can be user-friendly and accurately analyzed by clinicians to extend the use of MMC technology in the disease populations.

Introduction

Markerless motion capture (MMC) technology has been developed to avoid the need for marker placement during tracking and analyzing human movement [1]. By elimination of the time-consuming marker placement procedure, motion capturing experiment can be performed in a more convenient way [2]. Without the constraints that brought by body markers on movement, the development of MMC technology allows the capture of a more lifelike human motion in the environment, in a more natural way, and with the feature that it uses more portable and low-cost sensors compared to marker-based multi-camera systems [3], MMC in turn creates the potential of additional applications.

Previous studies have been conducted to compare the accuracy of MMC and body-marker-based analysis systems. Bonnechere et al. [4] compared the measuring accuracy of full body scanning by Microsoft Kinect 3D scanner software versus that of a high-resolution stereophotogrammetric system, which is a marker-based system in the healthy population. They concluded that Kinect is a reliable markerless tool that is suitable for use as a fast estimator of morphology. Schmitz et al. [5] validated the accuracy of Kinect in measuring knee joint angle of a jig by comparing its measurement using a digital inclinometer that acted as a ground-truth, and they reported that the performance of the Kinect system was satisfactory in terms of knee flexion and abduction. The accuracy of using a smartphone as a measurement system for joint angle has been reviewed by Mourcou et al. [6], who concluded that smartphone applications are reliable for clinical measurements of joint position and range of motion (ROM).

Earlier in 2006, Mündermann et al. [7] described several methods of MMC video processing modules including background separation, visual hull, and iterative closest point methods, etc., and pointed out that MMC has the potential to achieve a level of accuracy that facilitates the biomechanics research of normal and pathological human movement. Together with the reliable performance of MMC technology in the measurement of joint angle and body movement as reflected by [5, 6], it is suggested that the MMC system can be further applied to the rehabilitation field to measure patients’ motor function. However, the actual application of MMC technology for clinical measurement in rehabilitation is still at a preliminary stage. Most of the extant studies have focused on calibration of the MMC system or on validating the MMC system only on healthy persons. Applied research on the actual use of MMC technology in measurements in patient groups has been very diverse: Vivar and the teams [8] applied MMC technology in people with Parkinson’s disease (PD) to detect and classify their tremor level, while Gritsenko et al. [9] used Kinect as the MMC system to measure the shoulder ROM for women breast cancer patients after surgery. Instead of applying MMC technology in adults, Chin et al. [10] assessed the level of proprioceptive ability in children with cerebral palsy by using Kinect as the MMC system to measure the arm position of both healthy children and children with unilateral spastic cerebral palsy (USCP). These researchers found significant differences between the proprioceptive ability of the typically developing children and the children with USCP, as measured by Kinect, thus suggesting that MMC technology has the potential to be useful as a clinical measurement tool for proprioception.

Despite these trials, however, studies on the applications of MMC technology in clinical evaluation are still preliminary and limited in number, and it remains inconclusive how MMC technology can benefit therapists, patients, or the healthcare system, in terms of measuring patients’ conditions. Review studies have been conducted on the use of MMC technology in rehabilitation training, but not in regard to its use in clinical measurement including application of MMC technology in clinical assessment and detection of kinematic parameters that assist in disease diagnosis [11]. Mousavi Hondori and Khademi [12] reviewed the clinical impact of Kinect in rehabilitation, but their study did not cover other types of MMC technology. Therefore, to investigate the current uses of MMC technology as an assessment tool in the healthcare field, in this review we put less focus on the engineering components and attempted primarily to determine the current evidence for using MMC as a measurement tool, in order to further explore the potential benefits of MMC technology in rehabilitation evaluations. In this paper, we define clinical measurement as identification and measurement of movement kinematics in a clinical population [13], while MMC technology include systems and methods that capture and analysis movements without the need of marker placement, including video-based analysis. This systematic review further investigated: (1) the types of patients to whom MMC technology has been applied; (2) the contents of the MMC measurements; (3) the types of MMC systems used; and (4) the efficacy of these MMC systems as measurement tools.

Methods

Search strategy

A systematic computerized literature search was conducted by one of the authors (WTL) in PubMed, Medline, CINAHL, CENTRAL, EMBASE, and IEEE. Only peer-reviewed articles were included. The search keywords used in each database were “Markerless Motion Capture OR Motion Capture OR Motion Capture Technology OR Markerless Motion Capture Technology OR Computer Vision OR Video-based OR Pose Estimation AND Assessment OR Clinical Assessment OR Clinical Measurement OR Assess.” A manual search was also conducted that included searching Google Scholar using the same keywords, and the reference lists of the previous systematic reviews were also screened. The published data were not limited, and the last search took place on March 6, 2023.

Inclusion criteria

Studies were included if they met certain inclusion criteria. Specifically, the studies had to: (1) be peer-reviewed; (2) apply MMC technology for measurement; (3) involve subjects with symptomatic conditions; (4) have any quantitative study design except systematic reviews; (5) include at least one assessment item for clinical evaluation; and (6) be published in English.

Exclusion criteria

Studies were excluded if they met any one of the following exclusion criteria: (1) studying only healthy persons; (2) focusing only on calibration of the MMC system; (3) applying MMC technology only in rehabilitation training; or (4) not reporting results of an assessment evaluation.

Data extraction

The information we assessed included: (1) the types of MMC systems used in the studies; (2) the conditions of the participants that underwent the measurement, such as diagnoses or disabilities; and (3) the contents of the measurements conducted. The interpretations of the studies’ results were extracted and are presented in a summary table (Table 1). The contents of the measurement included the body functions or body parts that were measured, and the context in which the assessment was conducted.

Table 1 Details of the selected studies

Results

Literature search and study characteristics

A total of 4283 articles were identified, 278 of which were selected for full-text reading after removal of duplicates and irrelevancies, according to their abstracts (Fig. 1). After next excluding 213 articles on the basis of the inclusion and exclusion criteria, 65 studies remained and were included in the final review (Fig. 1). More than 40% of the studies applied MMC technology to assess a patient population with PD (n = 28) [8, 14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40]. Two other diseases that had commonly been measured by the MMC system were cerebral palsy (CP) (n = 6) [10, 41,42,43,44,45] and stroke (n = 6) [46,47,48,49,50,51]. Four other studies focused on children with autism spectrum disorder (ASD) (n = 4) [52,53,54,55] while there are two studies focused on patients with schizophrenia (n = 2) [56, 57] and patients with dementia (n = 2) [58, 59] respectively. The rest of the studies were conducted on various other diseases: Fragile X syndrome (FXS) [60], chronic neck pain [61], breast cancer [9], spinal cord injury (SCI) [62], amyotrophic lateral sclerosis (ALS) [63], adhesive capsulitis of shoulder (AC) [64], dystrophinopathy [65] and neuromotor diseases [66]. There were also studies that had been conducted on wheelchair users (n = 2) [67, 68], people awaiting total knee arthroplasty (TKR) [69], patients with gait disturbance [70], patients with neurodevelopment disorders (NDD) [71], patients with tremor [72], patients with Duchenne muscular dystrophy (DMD) [73], patients with cervical dystonia (CD) [74] and patients with a variety of diagnoses [75]. Table 1 summarizes the 65 selected studies.

Fig. 1
figure 1

Flow chart for selection of the studies for this review

Body function/body part being measured

Of the 28 studies that assessed the PD population by using MMC technology, fourteen measured the hand’s motor conditions to classify or to predict the severity of PD [8, 14,15,16,17,18,19,20,21,22,23,24, 39, 40]. These fourteen studies used the PD features of bradykinesia and tremor, as reflected during hand movements such as a finger-tapping exercise, to train machine-learning models to serve as classifiers. Of the remaining fourteen studies, four focused on using whole-body motion to classify PD [25,26,27, 38], and the other ten measured gait features to detect gait disorder in people with PD [28,29,30,31,32,33,34,35,36,37]. The measured body function for the CP population by the MMC system included gait pattern, trunk mobility, general body movement, fidgety movements, and the level of proprioceptive ability [10, 41,42,43,44,45]. The six studies on stroke survivors applied MMC technology to measure their upper limb movement, including their motor function, movement velocity, and joint angle [46,47,48,49] as well as lower limb movement gait parameters and walking pattern [50, 51]. The studies that worked on the ASD population mainly focused on prediction of diagnosis of ASD by children’s behavioral patterns [52,53,54,55]. The measured areas in the studies that applied MMC technology in patients with other types of diseases varied, and the details are listed in the summary table (Table 1).

Details of measurement and efficacy

The applications of the MMC systems in measurement were classified into several categories. Sixteen out of the 65 selected studies used MMC technology in symptoms identification in disease populations [8, 14, 15, 17, 21, 25, 30,31,32, 36, 39, 40, 50, 53, 54, 72]. Butt et al. attempted to distinguish patients with PD from healthy subjects by features of their hand movements, reporting that their Leap Motion Controller (LMC) system together with the machine-learning models did not provide a reliable measurement for the PD symptoms [15]. Fifteen studies focused on comparing the movement patterns of the disease populations and a healthy population, with all of them reporting a significant difference in at least one of the measured parameters including gait parameters, hand movement patterns, head movement patterns and general body movements [16, 20, 26, 28, 41, 49, 52, 55,56,57, 60, 61, 66, 70, 71]. Fifteen studies applied MMC technology to detect and identify movement limitations or specific movement patterns of patients with certain diseases, and significant parameters that indicate movement abnormity including bradykinesia, shuffling gait, abnormal walking pattern, and tremor were identified [9, 19, 24, 29, 33,34,35, 37, 42,43,44, 51, 59, 63, 73]. Two studies used the MMC system to measure range of motion (ROM), and they suggested MMC could be an alternative to the goniometer as a tool for ROM assessment [64, 75]. Three studies used the MMC system as a tool to analyze the wheelchair propulsion skills of wheelchair users [62, 67, 68]. Ten studies correlated or compared the MMC measurements with clinical assessment scales [18, 23, 27, 38, 46,47,48, 58, 65, 74]. Among the other three studies, one applied MMC technology in a comparison with the optic marker system [45], one used it to measure leg length [69], and one used it as a tool to assess proprioception [10]. Only one study reported unsatisfactory results, claiming that the use of the MMC system alone to measure leg length was not accurate [69]. The details are listed in the summary table (Table 1).

Types of MMC systems

Twenty studies used Kinect in their research, thus making Kinect the most popular MMC system used in the selected studies [9, 10, 30, 31, 43, 45,46,47,48,49, 58, 59, 62,63,64,65, 67, 68, 70, 75]. Sixteen studies used camera including RGB camera, digital video camera, GoPro camera and webcam [16,17,18, 20, 24, 25, 29, 32, 33, 35, 37, 41, 51, 54, 55, 71], while twelve studies analyzed patients’ movement by using smartphone or mobile tablet videos [14, 19, 21, 22, 27, 36, 38, 44, 50, 57, 72, 73]. Six studies performed the motion analysis from YouTube video or video recordings that captured by nonspecific capturing device [23, 34, 52, 53, 66, 74]. Five studies used the leap motion controller (LMC), an optical hand-tracking module [8, 15, 17, 39, 40]. The rest of the studies applied the BioStage™ System (Organic Motion Inc., N.Y., USA) (n = 3) [56, 60, 69]; the DARI Motion platform’s motion capture system (n = 1) [26]; the 4DBODY System (n = 1) [42], and a nonspecific customized motion capture system (n = 1) [61]. Table 2 describes and compares the characteristics of these seven types of MMC systems in terms of their mechanisms, set-up procedures, relative costs, the body part(s) that can be captured, and the systems’ methods of data extraction and analysis.

Table 2 Comparison of the MMC systems

Discussion

Our results revealed that most of the research applications of an MMC system for measurement were with patient groups with physical disabilities, and more than half of the studies assessed the PD and CP populations. A possible reason for this trend could be that both PD and CP have obvious and well-defined physical signs and symptoms and abnormal movements. PD is characterized by the presence of tremor, bradykinesia, and rigidity during movement [76], whereas patients with cerebral palsy demonstrate spasticity, ataxia, rigidity in movement, and the like [77]. The characteristic types of movement in these two groups of patients might be especially favorable for detection and analysis by the MMC system because of the significant homogeneity in the patients’ movement patterns. Applications of an MMC system for measurement in other kinds of physical disabilities have been limited, and that was the case in this review, but the heterogeneous disease types that were evaluated in the selected studies suggest the possibility of a high variety of generalized uses of MMC technology in assessing different types of patients.

In addition to the use of MMC systems in applications involving physical disabilities that demonstrate observable physical symptoms, it was noteworthy that such systems were also applied in patients with mental illness and NDD, in an attempt to deduce the presence of movement markers for mental disorder and the behavior associated with NDD. Experimental use of MMC technology in patients with mental illness and NDD suggests an entirely new trend for the application of MMC technology in the clinical field. Heretofore, motion tracking has been used in targeted patients with physical disabilities, because the analysis of their movements can provide necessary information and data about their level of impairment, and that in turn can indicate their recovery progress. However, although clinical observations have demonstrated that there is a difference between the movement patterns of patients with mental illness and those of healthy individuals, application of motion capture systems to assess the physical ability of patients with mental illness is still quite limited [78]. Since traditional marker-based systems for motion analysis are time-consuming to set up given that it requires calibration procedure and attachment of markers on the body, using the traditional motion analysis marker systems might not be cost-effective to study the motion dysfunctions in patients with mental illness whose cognitive functions are predominantly affected. In fact, previous studies on motion detection of patients with mental illness adopted the fuzzy movement method, and precise actions and movement patterns have been less emphasized [79]. Therefore, the development of MMC technology allows motion capture in a more cost-effective way, and that improvement may facilitate future scientific investigations of movement patterns and motor functions in patients with mental illness. Identifying the risk of NDD by extracting the children’s behavioral features with the help of computer-vision technology also proposed a new direction of early screening of NDD [80], in which children’s developmental conditions can be closely monitored in their familiar environment without the need of attachment of markers on the infants’ body. Similarly, the studies that have applied the MMC system to compare the motion patterns of a disease population and a healthy population provide evidence for the technology’s use to identify biomarkers for certain diseases. MMC technology may also contribute to the development and use of big data for future AI screening for diseases, based on body movements. The combination of MMC technology and a machine-learning algorithm in classification of CP in infants by Nguyen-Thai et al. [44] is one of the good examples that demonstrates how MMC technology can help in the preliminary screening of diseases. Compared with screening methods for traditional diseases, which depend heavily on behavioral observations by parents or on subjective self-reported questionnaires [81], MMC technology, which identifies symptoms via movement detection, could be a more objective method for early screening for diseases, facilitating early identification of a disease and thus improving the prognosis for rehabilitation, as well as providing a tool for evaluation before and after rehabilitation.

In contrast to using MMC technology for symptoms identification or for detection of differences in movement patterns between disease groups and their healthy counterparts, other studies applied MMC technology as a direct clinical measurement tool. Although the use of the MMC system to measure leg length was found to be inaccurate [69], the use of Kinect to measure ROM was found to be reliable [64, 75]. These findings suggest that MMC technology might have the potential to serve as an alternative clinical assessment tool. MMC technology also provides a new approach to assessing functional or cognitive abilities, such as objectively evaluating proprioception, which previously relied heavily on manual evaluations by rehabilitation therapists. However, future studies on the measurement accuracy and the validity of MMC technology as a clinical measurement tool are warranted.

Microsoft Kinect, the most commonly used MMC system in the studies in this review, is a relatively low-cost, commercially available system that captures and analyzes whole-body movement. Kinect enables the capture of real-time whole body gross movements, but it appears to be less sensitive in tracking fine hand movements [82]. Although many of the studies used Kinect in their MMC measurements, the system has been out of production since 2017 and was no longer supported by the Xbox Series X, as announced by Microsoft [83]. Future rehabilitation assessors that wish to use MMC technology may have to consider using other kinds of MMC systems, or the newly developed Azure Kinect. Our review also found that the most recent studies adopted the use of camera, smartphone, or video clips from the internet in conjunction with pose estimation algorithms and motion analysis algorithm, which has been rapidly developed in the recent years, to capture images and analyze motion. Human pose estimation method is a way of identifying and classifying human joints position using computer vision, for example, the open-source libraries OpenPose and PoseNet for human pose estimation are widely adopted in motion analysis [84]. With the development of human pose estimation database containing various types of movement datasets, accuracy of pose estimation from video clips can be further enhanced by using a large set of training data. This facilitates the use computer vision methods for motion analysis in video clips captured by portable and low-cost camera rather than using specific sensors in the traditional way. The use of machine-learning algorithms allows meaningful information such as kinematic data to be extracted directly from regular videos, thus making the use of MMC technology much easier in motion capturing in a natural environment without the need to buy any extra hardware devices. Human pose estimation technology such as Convolutional Pose Machines (CPM) and convolution neural network (CNN) based methods which allow extraction of human movement information directly from video clips have been repeatedly tested by researchers [85, 86] while human pose estimation application on analyzing movement in the disease populations were reported to be useful by the studies in our review [14, 16,17,18,19,20,21,22,23,24,25, 27, 29, 32,33,34,35,36,37,38, 41, 44, 50,51,52,53,54,55, 57, 66, 71,72,73,74]. Given that such trajectory extraction method is in rapid evaluation and is becoming more mature for promising identification of posture [87,88,89], using hand-held camera or smartphone as the MMC system would be especially beneficial for understanding the motor performance of individuals in their daily living tasks, hence providing valuable information on levels of impairment and on the constraints that patients might encounter in their activities of daily living in their real-life environment. It is understandable that individuals, particularly young children and older people, might behave differently when they are placed for motion capturing in an unfamiliar laboratory or a simulated environment, thus risking the possibility that the motion analysis might not truly reflect the individuals’ actual movement patterns [90]. The use of a smartphone camera combined with an algorithm for analysis could provide a solution to that problem and suggests the feasibility of assessing patients’ daily movements through an MMC combination of a smartphone and an advanced algorithm. Since it does not require additional hardware for motion capturing, such a system would further broaden MMC technology for measurement and clinical assessment in the field of rehabilitation.

Limitations of the current MMC technology’s applications for clinical measurement

Although the use of MMC system in motion capturing is becoming more common in movement measurement and helps us extend the application of MMC technology to clinical use, the technologies used for analyzing movement and distinguishing motor patterns are not yet generalized. Extracting and processing the data from MMC devices video files is still complicated and time-consuming, preventing the approach from being user-friendly for therapists to adopt as a quick clinical measurement tool. Further investigation is needed in order to design and develop a platform or software that can accurately analyze the movement patterns from videos in a more user-friendly and accurately way so as to further extend its application by clinicians. Although most of the studies that we included reported detecting a significant difference between the motor parameters of healthy control groups and those of disease populations, and while the identification of physical symptoms by the MMC system was also reported to be mostly effective, the sample sizes adopted by these studies were too small. A reliable AI tool for disease screening and classification will need to be trained and tested from a large set of data, to provide better specificity and sensitivity. In order to make use of MMC technology-assisted AI screening and early detection of diseases, a larger database that records movement patterns of both the disease population and the healthy population must be developed. Research on the development and selection of a suitable machine-learning or deep-learning model for classification is also needed. Ultimately, a cost-effective and accurate method for early patient screening will help therapists to identify individuals at risk and involve them in further, in-depth assessments, so that subsequent interventions can be made as early as possible. Moreover, it has been suggested that telerehabilitation could incorporate the use of MMC technology as a measurement tool for assessing and monitoring patients’ prognosis and recovery, thus offering an objective and precise evaluation of patients’ rehabilitation progress.

Conclusions

This review explored the current uses of MMC technology to perform assessments in clinical situations. Most of the studies in the review combined MMC technology and a classification algorithm to perform symptoms identification for disease populations or to detect the differences in movement between disease groups and their healthy counterparts. Findings from these studies revealed a potential use of MMC technology for detecting and identifying disease signs and symptoms. The method might also contribute to early screening by using AI and big data to screen for diseases that lead to physical or mental disabilities. Further studies are warranted to develop and integrate MMC system in a platform that can be user-friendly and accurately analyzed by clinicians to extend the use of MMC technology in clinical measurement.

Availability of data and materials

Not applicable.

References

  1. Corazza S, Mündermann L, Gambaretto E, Ferrigno G, Andriacchi TP. Markerless motion capture through visual hull, articulated icp and subject specific model generation. Int J Comput Vision. 2010;87(1):156–69.

    Article  Google Scholar 

  2. Rahul M. Review on motion capture technology. Glob J Comput Sci Technol. 2018;18(F1):23–6.

    Google Scholar 

  3. Scott B, Seyres M, Philp F, Chadwick EK, Blana D. Healthcare applications of single camera markerless motion capture: a scoping review. PeerJ. 2022;10:e13517.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Bonnechere B, Jansen B, Salvia P, Bouzahouene H, Sholukha V, Cornelis J, et al. Determination of the precision and accuracy of morphological measurements using the Kinect™ sensor: comparison with standard stereophotogrammetry. Ergonomics. 2014;57(4):622–31.

    Article  CAS  PubMed  Google Scholar 

  5. Schmitz A, Ye M, Shapiro R, Yang R, Noehren B. Accuracy and repeatability of joint angles measured using a single camera markerless motion capture system. J Biomech. 2013;47(2):587–91.

    Article  PubMed  Google Scholar 

  6. Mourcou Q, Fleury A, Diot B, Franco C, Vuillerme N. Mobile phone-based joint angle measurement for functional assessment and rehabilitation of proprioception. Biomed Res Int. 2015;2015:328142–15.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Mündermann L, Corazza S, Chaudhari AM, Andriacchi TP, Sundaresan A, Chellappa R, editors. Measuring human movement for biomechanical applications using markerless motion capture. Three-dimensional image capture and applications VII; 2006: International Society for Optics and Photonics.

  8. Vivar G, Almanza-Ojeda D-L, Cheng I, Gomez JC, Andrade-Lucio JA, Ibarra-Manzano M-A. Contrast and homogeneity feature analysis for classifying tremor levels in Parkinson’s disease patients. Sensors (Basel). 2019;19(9):2072.

    Article  PubMed  Google Scholar 

  9. Gritsenko V, Dailey E, Kyle N, Taylor M, Whittacre S, Swisher AK. Feasibility of using low-cost motion capture for automated screening of shoulder motion limitation after breast cancer surgery. PLoS ONE. 2015;10(6):e0128809.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Chin K, Soles L, Putrino D, Dehbandi B, Nwankwo V, Gordon A, et al. Use of markerless motion capture to evaluate proprioception impairments in children with unilateral spastic cerebral palsy: a feasibility trial. Dev Med Child Neurol. 2017;59:24–5.

    Article  Google Scholar 

  11. Knippenberg E, Verbrugghe J, Lamers I, Palmaers S, Timmermans A, Spooren A. Markerless motion capture systems as training device in neurological rehabilitation: a systematic review of their use, application, target population and efficacy. J Neuroeng Rehab. 2017;14(1):1–11.

    Article  Google Scholar 

  12. Mousavi Hondori H, Khademi M. A review on technical and clinical impact of microsoft kinect on physical therapy and rehabilitation. J Med Eng. 2014;2014:846514.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Sakkos D, Mccay KD, Marcroft C, Embleton ND, Chattopadhyay S, Ho ESL. Identification of abnormal movements in infants: a deep neural network for body part-based prediction of cerebral palsy. IEEE Access. 2021;9:94281–92.

    Article  Google Scholar 

  14. Khan T, Nyholm D, Westin J, Dougherty M. A computer vision framework for finger-tapping evaluation in Parkinson’s disease. Artif Intell Med. 2013;60(1):27–40.

    Article  PubMed  Google Scholar 

  15. Butt AH, Rovini E, Dolciotti C, De Petris G, Bongioanni P, Carboncini MC, et al. Objective and automatic classification of Parkinson disease with leap motion controller. Biomed Eng Online. 2018;17(1):168.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  16. Langevin R, Ali MR, Sen T, Snyder C, Myers T, Dorsey ER, et al. The PARK framework for automated analysis of Parkinson’s disease characteristics. Proc ACM Interact Mob Wearable Ubiquitous Technol. 2019;3(2):Article 54.

    Article  Google Scholar 

  17. Lee WL, Sinclair NC, Jones M, Tan JL, Proud EL, Peppard R, et al. Objective evaluation of bradykinesia in Parkinson’s disease using an inexpensive marker-less motion tracking system. Physiol Meas. 2019;40(1):014004.

    Article  PubMed  Google Scholar 

  18. Liu Y, Chen J, Hu C, Ma Y, Ge D, Miao S, et al. Vision-based method for automatic quantification of Parkinsonian Bradykinesia. IEEE Trans Neural Syst Rehab Eng. 2019;27(10):1952–61.

    Article  Google Scholar 

  19. Lin B, Luo W, Luo Z, Wang B, Deng S, Yin J, et al. Bradykinesia recognition in Parkinson’s disease via single RGB video. ACM Trans Knowl Discov Data. 2020;14(2):Article 16.

    Article  Google Scholar 

  20. Pang Y, Christenson J, Jiang F, Lei T, Rhoades R, Kern D, et al. Automatic detection and quantification of hand movements toward development of an objective assessment of tremor and bradykinesia in Parkinson’s disease. J Neurosci Methods. 2020;333:108576.

    Article  PubMed  Google Scholar 

  21. Williams S, Relton SD, Fang H, Alty J, Qahwaji R, Graham CD, et al. Supervised classification of bradykinesia in Parkinson’s disease from smartphone videos. Artif Intell Med. 2020;110:101966.

    Article  PubMed  Google Scholar 

  22. Williams S, Zhao Z, Hafeez A, Wong DC, Relton SD, Fang H, et al. The discerning eye of computer vision: can it measure Parkinson’s finger tap bradykinesia? J Neurol Sci. 2020;416:117003.

    Article  PubMed  Google Scholar 

  23. Li H, Shao X, Zhang C, Qian X. Automated assessment of parkinsonian finger-tapping tests through a vision-based fine-grained classification model. Neurocomputing (Amsterdam). 2021;441:260–71.

    Article  Google Scholar 

  24. Guo Z, Zeng W, Yu T, Xu Y, Xiao Y, Cao X, et al. Vision-based finger tapping test in patients with Parkinson’s disease via spatial-temporal 3D hand pose estimation. IEEE J Biomed Health Inform. 2022;26(8):3848–59.

    Article  PubMed  Google Scholar 

  25. Li MH, Mestre TA, Fox SH, Taati B. Vision-based assessment of parkinsonism and levodopa-induced dyskinesia with pose estimation. J Neuroeng Rehab. 2018;15(1):1–13.

    Article  Google Scholar 

  26. Martinez HR, Garcia-Sarreon A, Camara-Lemarroy C, Salazar F, Guerrero-González ML. Accuracy of Markerless 3D motion capture evaluation to differentiate between On/Off status in Parkinson’s disease after deep brain stimulation. Parkinsons Dis. 2018;2018:5830364.

    PubMed  PubMed Central  Google Scholar 

  27. Morinan G, Dushin Y, Sarapata G, Rupprechter S, Peng Y, Girges C, et al. Computer vision quantification of whole-body parkinsonian bradykinesia using a large multi-site population. NPJ Parkinson’s Dis. 2023;9(1):10.

    Article  Google Scholar 

  28. Cho C-W, Chao W-H, Lin S-H, Chen Y-Y. A vision-based analysis system for gait recognition in patients with Parkinson’s disease. Expert Systems with applications. 2009;36(3):7033–9.

    Article  Google Scholar 

  29. Chen S-W, Lin S-H, Liao L-D, Lai H-Y, Pei Y-C, Kuo T-S, et al. Quantification and recognition of parkinsonian gait from monocular video imaging using kernel-based principal component analysis. Biomed Eng Online. 2011;10(1):99.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Tupa O, Prochazka A, Vysata O, Schaetz M, Mares J, Valis M, et al. Motion tracking and gait feature estimation for recognising Parkinson’s disease using MS Kinect. Biomed Eng Online. 2015;14(1):97.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Dranca L, de Abetxuko Ruiz de Mendarozketa L, Goñi A, Illarramendi A, Navalpotro Gomez I, Delgado Alvarado M, et al. Using Kinect to classify Parkinson’s disease stages related to severity of gait impairment. BMC Bioinform. 2018;19(1):471.

    Article  Google Scholar 

  32. Li T, Chen J, Hu C, Ma Y, Wu Z, Wan W, et al. Automatic timed Up-and-Go Sub-Task segmentation for Parkinson’s disease patients using video-based activity classification. IEEE Trans Neural Sys Rehab Eng. 2018;26(11):2189–99.

    Article  Google Scholar 

  33. Sato K, Nagashima Y, Mano T, Iwata A, Toda T. Quantifying normal and parkinsonian gait features from home movies: practical application of a deep learning–based 2D pose estimator. PLoS ONE. 2019;14(11):e0223549.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  34. Hu K, Wang Z, Mei S, Martens KAE, Yao T, Lewis SJG, et al. Vision-based freezing of gait detection with anatomic directed graph representation. IEEE J Biomed Health Inform. 2020;24(4):1215–25.

    Article  PubMed  Google Scholar 

  35. Cao X, Xue Y, Chen J, Chen X, Ma Y, Hu C, et al. Video based shuffling step detection for parkinsonian patients using 3d convolution. IEEE Trans Neural Syst Rehab Eng. 2021;29:641–9.

    Article  Google Scholar 

  36. Rupprechter S, Morinan G, Peng Y, Foltynie T, Sibley K, Weil RS, et al. A clinically interpretable computer-vision based method for quantifying gait in parkinson’s disease. Sensors. 2021;21(16):5437.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Stricker M, Hinde D, Rolland A, Salzman N, Watson A, Almonroeder TG. Quantifying step length using two-dimensional video in individuals with Parkinson’s disease. Physiother Theory Pract. 2021;37(1):252–5.

    Article  PubMed  Google Scholar 

  38. Morinan G, Peng Y, Rupprechter S, Weil RS, Leyland L-A, Foltynie T, et al. Computer-vision based method for quantifying rising from chair in Parkinson’s disease patients. Intelligence-Based Medicine. 2022;6:100046.

    Article  Google Scholar 

  39. Oña ED, Jardón A, Cuesta-Gómez A, Sánchez-Herrera-Baeza P, Cano-de-la-Cuerda R, Balaguer C. Validity of a fully-immersive VR-based version of the box and blocks test for upper limb function assessment in Parkinson’s disease. Sensors. 2020;20(10):2773.

    Article  PubMed  PubMed Central  Google Scholar 

  40. Wu J, Yu N, Yu Y, Li H, Wu F, Yang Y, et al. Intraoperative quantitative measurements for Bradykinesia evaluation during deep brain stimulation surgery using Leap Motion Controller: a pilot study. Parkinson’s Disease. 2021;2021:6639762.

    PubMed  PubMed Central  Google Scholar 

  41. Adde L, Helbostad JL, Jensenius AR, Taraldsen G, Grunewaldt KH, StØen R. Early prediction of cerebral palsy by computer-based video analysis of general movements: a feasibility study. Dev Med Child Neurol. 2010;52(8):773–8.

    Article  PubMed  Google Scholar 

  42. Krasowicz K, Michoński J, Liberadzki P, Sitnik R. Monitoring improvement in infantile cerebral palsy patients using the 4DBODY system—a preliminary study. Sensors (Basel). 2020;20(11):3232.

    Article  PubMed  Google Scholar 

  43. Schroeder AS, Hesse N, Weinberger R, Tacke U, Gerstl L, Hilgendorff A, et al. General Movement Assessment from videos of computed 3D infant body models is equally effective compared to conventional RGB video rating. Early Hum Dev. 2020;144:104967.

    Article  PubMed  Google Scholar 

  44. Nguyen-Thai B, Le V, Morgan C, Badawi N, Tran T, Venkatesh S. A spatio-temporal attention-based model for Infant Movement Assessment from videos. IEEE J Biomed Health Inform. 2021;25(10):3911–20.

    Article  PubMed  Google Scholar 

  45. Pantzar-Castilla E, Cereatti A, Figari G, Valeri N, Paolini G, Della Croce U, et al. Knee joint sagittal plane movement in cerebral palsy: a comparative study of 2-dimensional markerless video and 3-dimensional gait analysis. Acta Orthop. 2018;89(6):656–61.

    Article  PubMed  PubMed Central  Google Scholar 

  46. Olesh EV, Yakovenko S, Gritsenko V. Automated assessment of upper extremity movement impairment due to stroke. PloS one. 2014;9(8):e104487-e.

    Article  Google Scholar 

  47. Kim W-S, Cho S, Baek D, Bang H, Paik N-J. Upper Extremity Functional evaluation by Fugl-Meyer Assessment Scoring using depth-sensing camera in hemiplegic stroke patients. PLoS One. 2016;11(7):e0158640-e.

    Article  Google Scholar 

  48. Bakhti KKA, Laffont I, Muthalib M, Froger J, Mottet D. Kinect-based assessment of proximal arm non-use after a stroke. J Neuroeng Rehabil. 2018;15(1):104.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  49. Bonnechère B, Sholukha V, Omelina L, Van Sint Jan S, Jansen B. 3D analysis of upper limbs motion during rehabilitation exercises using the KinectTM sensor: development, laboratory validation and clinical application. Sensors. 2018;18(7):2216.

    Article  PubMed  PubMed Central  Google Scholar 

  50. Lee JT, Park E, Jung T-D. Machine learning-based classification of dependence in ambulation in stroke patients using smartphone video data. J Personalized Med. 2021;11(11):1080.

    Article  Google Scholar 

  51. Lonini L, Moon Y, Embry K, Cotton RJ, McKenzie K, Jenz S, et al. Video-based pose estimation for gait analysis in stroke survivors during clinical assessments: a proof-of-concept study. Digital Biomarkers. 2022;6(1):9–18.

    Article  PubMed  PubMed Central  Google Scholar 

  52. Caruso A, Gila L, Fulceri F, Salvitti T, Micai M, Baccinelli W, et al. Early Motor Development predicts clinical outcomes of siblings at high-risk for Autism: insight from an innovative motion-tracking technology. Brain Sci. 2020;10(6):379.

    Article  PubMed  PubMed Central  Google Scholar 

  53. Negin F, Ozyer B, Agahian S, Kacdioglu S, Ozyer GT. Vision-assisted recognition of stereotype behaviors for early diagnosis of Autism Spectrum Disorders. Neurocomputing. 2021;446:145–55.

    Article  Google Scholar 

  54. Song C, Wang S, Chen M, Li H, Jia F, Zhao Y. A multimodal discrimination method for the response to name behavior of autistic children based on human pose tracking and head pose estimation. Displays. 2023;76:102360.

    Article  Google Scholar 

  55. Kojovic N, Natraj S, Mohanty SP, Maillart T, Schaer M. Using 2D video-based pose estimation for automated prediction of autism spectrum disorders in young children. Scientific Reports. 2021;11(1):1–10.

    Article  Google Scholar 

  56. Sá F, Marques A, Rocha NBF, Trigueiro MJ, Campos C, Schröder J. Kinematic parameters of throwing performance in patients with schizophrenia using a markerless motion capture system. Somatosens Mot Res. 2015;32(2):77–86.

    Article  PubMed  Google Scholar 

  57. Abbas A, Yadav V, Smith E, Ramjas E, Rutter SB, Benavidez C, et al. Computer vision-based assessment of motor functioning in schizophrenia: use of smartphones for remote measurement of schizophrenia symptomatology. Digital Biomarkers. 2021;5(1):29–36.

    Article  PubMed  PubMed Central  Google Scholar 

  58. Sabo A, Mehdizadeh S, Ng K-D, Iaboni A, Taati B. Assessment of Parkinsonian gait in older adults with dementia via human pose tracking in video data. Journal of neuroengineering and rehabilitation. 2020;17(1):1–10.

    Article  Google Scholar 

  59. Mehdizadeh S, Faieghi M, Sabo A, Nabavi H, Mansfield A, Flint AJ, et al. Gait changes over time in hospitalized older adults with advanced dementia: predictors of mobility change. PLoS ONE. 2021;16(11):e0259975-e.

    Article  Google Scholar 

  60. O’Keefe JA, Orías AAE, Khan H, Hall DA, Berry-Kravis E, Wimmer MA. Implementation of a markerless motion analysis method to quantify hyperkinesis in males with fragile X syndrome. Gait Posture. 2013;39(2):827–30.

    Article  PubMed  Google Scholar 

  61. Bahat HSPPT, Weiss PLPOT, Laufer YDPT. The Effect of Neck Pain on Cervical Kinematics, as assessed in a virtual environment. Arch Phys Med Rehabil. 2010;91(12):1884–90.

    Article  Google Scholar 

  62. Zefinetti FC, Vitali A, Regazzoni D, Rizzi C, Molinero G. Tracking and characterization of spinal cord-injured patients by means of rgb-d sensors. Sensors (Basel). 2020;20(21):1–20.

    Google Scholar 

  63. de Bie E, Oskarsson B, Joyce NC, Nicorici A, Kurillo G, Han JJ. Longitudinal evaluation of upper extremity reachable workspace in ALS by Kinect sensor. Amyotroph Lateral Scler Frontotemporal Degener. 2017;18(1–2):17–23.

    Article  PubMed  Google Scholar 

  64. Lee SH, Yoon C, Chung SG, Kim HC, Kwak Y, Park H-W, et al. Measurement of shoulder range of motion in patients with Adhesive Capsulitis using a Kinect. PLoS One. 2015;10(6):e0129398-e.

    Article  Google Scholar 

  65. Lowes LP, Alfano LN, Yetter BA, Worthen-Chaudhari L, Hinchman W, Savage J, et al. Proof of concept of the ability of the kinect to quantify upper extremity function in dystrophinopathy. PLoS Curr. 2013. https://doi.org/10.1371/currents.md.9ab5d872bbb944c6035c9f9bfd314ee2.

    Article  PubMed  PubMed Central  Google Scholar 

  66. Chambers C, Seethapathi N, Saluja R, Loeb H, Pierce SR, Bogen DK, et al. Computer vision to automatically assess infant Neuromotor Risk. IEEE Trans Neural Syst Rehabil Eng. 2020;28(11):2431–42.

    Article  PubMed  PubMed Central  Google Scholar 

  67. Rammer J, Slavens B, Krzak J, Winters J, Riedel S, Harris G. Assessment of a markerless motion analysis system for manual wheelchair application. J Neuroeng Rehabil. 2018;15(1):96.

    Article  PubMed  PubMed Central  Google Scholar 

  68. Wei L, Chung C-S, Koontz AM. Automating the Clinical Assessment of Independent Wheelchair sitting pivot transfer techniques. Topics Spinal Cord injury Rehab. 2021;27(3):1–11.

    Article  Google Scholar 

  69. Hurley RJ, Davey MS, Newell M, Devitt A. Assessing the accuracy of measuring leg length discrepancy and genu varum/valgum using a markerless motion analysis system. J Orthop. 2021;26:45–8.

    Article  PubMed  PubMed Central  Google Scholar 

  70. Fujii M, Wada N, Ikeda Y, Hasegawa M, Nakazato S, Yuminaka Y, et al. Rehabilitation Assistance Systems for three-dimensional gait analysis using motion capture Devices. Advanced engineering forum. 2020;38:209–14.

    Article  Google Scholar 

  71. Ardalan A, Yamane N, Rao AK, Montes J, Goldman S. Analysis of gait synchrony and balance in neurodevelopmental disorders using computer vision techniques. Health Informatics Journal. 2021;27(4):14604582211055650.

    Article  PubMed  Google Scholar 

  72. Williams S, Fang H, Relton SD, Wong DC, Alam T, Alty JE. Accuracy of smartphone video for contactless measurement of hand tremor frequency. Movement Disorders Clinical Practice. 2021;8(1):69–75.

    Article  PubMed  Google Scholar 

  73. Ferrer-Mallol E, Matthews C, Stoodley M, Gaeta A, George E, Reuben E, et al. Patient-led development of digital endpoints and the use of computer vision analysis in assessment of motor function in rare diseases. Front Pharmacol. 2022;13:916714.

    Article  PubMed  PubMed Central  Google Scholar 

  74. Vu JP, Cisneros E, Lee HY, Le L, Chen Q, Guo XA, et al. Head tremor in cervical dystonia: quantifying severity with computer vision. Journal of the Neurological Sciences. 2022;434:120154.

    Article  PubMed  PubMed Central  Google Scholar 

  75. Matsen FAMD, Lauder AMD, Rector KMS, Keeling PMD, Cherones AL. Measurement of active shoulder motion using the Kinect, a commercially available infrared position detection system. J Shoulder Elbow Surg. 2016;25(2):216–23.

    Article  PubMed  Google Scholar 

  76. Poewe W, Seppi K, Tanner CM, Halliday GM, Brundin P, Volkmann J, et al. Parkinson disease. Nat Rev Dis Primers. 2017;3(1):1–21.

    Article  Google Scholar 

  77. Rosenbaum P, Paneth N, Levinton A, Goldstein M, Bax M, Damiano D, et al. The definition and classification of cerebral palsy. NeoReviews. 2006;7(11):e569.

    Article  Google Scholar 

  78. Walther S, van Harten PN, Waddington JL, Cuesta MJ, Peralta V, Dupin L, et al. Movement disorder and sensorimotor abnormalities in schizophrenia and other psychoses-european consensus on assessment and perspectives. Eur Neuropsychopharmacol. 2020;38:25–39.

    Article  CAS  PubMed  Google Scholar 

  79. Walther S, Ramseyer F, Horn H, Strik W, Tschacher W. Less structured movement patterns predict severity of positive syndrome, excitement, and disorganization. Schizophrenia Bull. 2014;40(3):585–91.

    Article  Google Scholar 

  80. de Belen RAJ, Bednarz T, Sowmya A, Del Favero D. Computer vision in autism spectrum disorder research: a systematic review of published studies from 2009 to 2019. Transl Psychiatry. 2020;10(1):333.

    Article  PubMed  PubMed Central  Google Scholar 

  81. Horwitz E, Schoevers R, Ketelaars C, Kan C, Van Lammeren A, Meesters Y, et al. Clinical assessment of ASD in adults using self-and other-report: psychometric properties and validity of the adult Social Behavior Questionnaire (ASBQ). Research in Autism Spectrum Disorders. 2016;24:17–28.

    Article  Google Scholar 

  82. Galna B, Barry G, Jackson D, Mhiripiri D, Olivier P, Rochester L. Accuracy of the Microsoft Kinect sensor for measuring movement in people with Parkinson’s disease. Gait & posture. 2014;39(4):1062–8.

    Article  Google Scholar 

  83. Weinberger M. The rise and fall of Kinect: why Microsoft gave up on its most promising product. Bussinessinsider. 2018.

  84. Nishani E, Çiço B, editors. Computer vision approaches based on deep learning and neural networks: Deep neural networks for video analysis of human pose estimation. 2017 6th Mediterranean Conference on Embedded Computing (MECO); 2017: IEEE.

  85. Qiang B, Zhang S, Zhan Y, Xie W, Zhao T. Improved convolutional pose machines for human pose estimation using image sensor data. Sensors. 2019;19(3):718.

    Article  PubMed  PubMed Central  Google Scholar 

  86. Andrade-Ambriz YA, Ledesma S, Ibarra-Manzano M-A, Oros-Flores MI, Almanza-Ojeda D-L. Human activity recognition using temporal convolutional neural network architecture. Expert Systems with Applications. 2022;191:116287.

    Article  Google Scholar 

  87. Wrench A, Balch-Tomes J. Beyond the edge: markerless pose estimation of speech articulators from ultrasound and camera images using DeepLabCut. Sensors. 2022;22(3):1133.

    Article  PubMed  PubMed Central  Google Scholar 

  88. Doosti B, Naha S, Mirbagheri M, Crandall DJ, editors. Hope-net: A graph-based model for hand-object pose estimation. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition; 2020.

  89. Luo Y, Ou Z, Wan T, Guo J-M. FastNet: Fast high-resolution network for human pose estimation. Image and Vision Computing. 2022;119:104390.

    Article  Google Scholar 

  90. Tronick E, Als H, Brazelton T. Early development of neonatal and infant behavior. Human growth. Berlin: Springer; 1979. p. 305–28.

    Google Scholar 

Download references

Funding

This research project was funded by Research Impact Fund (Ref. no.: R5028-20 F) to KNKF, Research Grants Council, University Grants Committee, Hong Kong SAR.

Author information

Authors and Affiliations

Authors

Contributions

WWTL, KNKF prepared the study objectives. WWTL did the literature search. WWTL and YMT did the data extraction and screening. WWTL and YMT did the methodological quality assessment. WWTL, YMT and KNKF wrote the main manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Kenneth N. K. Fong.

Ethics declarations

Consent for publication

All authors have approved this manuscript for publication. It has not been previously published, nor is it pending publication elsewhere.

Competing interests

The author(s) declare no potential conflicts of interest with respect to this article’ research, authorship, and/or publication that might be perceived to influence the results and/ or discussion reported in this paper.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lam, W.W.T., Tang, Y.M. & Fong, K.N.K. A systematic review of the applications of markerless motion capture (MMC) technology for clinical measurement in rehabilitation. J NeuroEngineering Rehabil 20, 57 (2023). https://doi.org/10.1186/s12984-023-01186-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12984-023-01186-9

Keywords