Autonomous identification of freezing of gait in Parkinson's disease from lower-body segmental accelerometry
© Moore et al.; licensee BioMed Central Ltd. 2013
Received: 7 March 2012
Accepted: 5 February 2013
Published: 13 February 2013
We have previously published a technique for objective assessment of freezing of gait (FOG) in Parkinson's disease (PD) from a single shank-mounted accelerometer. Here we extend this approach to evaluate the optimal configuration of sensor placement and signal processing parameters using seven sensors attached to the lumbar back, thighs, shanks and feet.
Multi-segmental acceleration data was obtained from 25 PD patients performing 134 timed up and go tasks, and clinical assessment of FOG was performed by two experienced raters from video. Four metrics were used to compare objective and clinical measures; the intraclass correlation coefficient (ICC) for number of FOG episodes and the percent time frozen per trial; and the sensitivity and specificity of FOG detection.
The seven-sensor configuration was the most robust, scoring highly on all measures of performance (ICC number of FOG 0.75; ICC percent time frozen 0.80; sensitivity 84.3%; specificity 78.4%). A simpler single-shank sensor approach provided similar ICC values and exhibited a high sensitivity to FOG events, but specificity was lower at 66.7%. Recordings from the lumbar sensor offered only moderate agreement with the clinical raters in terms of absolute number and duration of FOG events (likely due to musculoskeletal attenuation of lower-limb 'trembling' during FOG), but demonstrated a high sensitivity (86.2%) and specificity (82.4%) when considered as a binary test for the presence/absence of FOG within a single trial.
The seven-sensor approach was the most accurate method for quantifying FOG, and is best suited to demanding research applications. A single shank sensor provided measures comparable to the seven-sensor approach but is relatively straightforward in execution, facilitating clinical use. A single lumbar sensor may provide a simple means of objective FOG detection given the ubiquitous nature of accelerometers in mobile telephones and other belt-worn devices.
KeywordsFOG Timed up-and-go task Accelerometer
Freezing of gait (FOG), a paroxysmal block of movement when initiating gait, turning, or negotiating an obstacle, is often described by the Parkinson's disease (PD) patient as the sensation that their feet are 'stuck to the ground'. FOG is generally regarded as a late feature of PD associated with disease duration and severity [1, 2], although some early-stage PD patients (patients not yet administered levodopa [2, 3] or with mild symptoms ) also experience freezing. FOG is associated with an increased prevalence of falls, loss of independence, and nursing home placement .
Clinical management of FOG is limited in large part by the difficult nature of assessing its severity, and subjective measures have dominated the field. The Unified Parkinson’s Disease Rating Scale part 14 'Freezing When Walking' (UPDRS 14)  rates patients on an ordinal scale from 0 (none) to 4 (frequent falls from freezing) based on clinical history. This was the primary outcome measure (UPDRS 14 ≥1) used in a large study of selegiline as a prophylactic treatment for FOG in early PD . A Freezing of Gait Questionnaire (FOG-Q)  and the new FOG-Q (NFOG-Q)  have recently been proposed as more sensitive tools to identify FOG behavior and assess the efficacy of interventions . However, neither the FOG-Q nor NFOG-Q score correlated with the severity (frequency or duration) of freezing episodes during actual walking in our recent study of PD patients with self-reported FOG . Clinical evaluation of video recordings of ambulating patients to identify the number of FOG events utilizing one [11–15], two [10, 15–19], or three observers [20–23] has emerged as a de facto 'gold standard' in the past decade. Our recent study utilizing 10 experienced raters across four leading PD centers assessing videos from 'freezers' found only moderate inter-rater agreement for number of FOG events (intraclass correlation coefficient, ICC=0.6), and intra-rater reliability was remarkably low (mean ICC=0.4) .
Current treatment options for FOG are largely ineffective . There is an increased prevalence of freezing in advanced disease and in the clinical 'off' state, highlighting the key role of striatal dopamine depletion in its pathogenesis . Increasing levodopa dosage can reduce the frequency of 'off' (unmedicated) state freezing [1, 22], likely by increasing the threshold for FOG to occur without altering the underlying pathophysiology . However, FOG commonly shows only partial response to levodopa , and the benefits of increasing levodopa dosage in reducing FOG must be balanced with the increased likelihood of levodopa-induced dyskinesias, also associated with a greater fall risk . Patients may undergo deep brain stimulation surgery to relieve symptoms of 'off' state FOG, although these surgical interventions are currently viewed as a treatment option only in the latter stages of PD . Clinical management of dopaminergic therapy to minimize FOG, as well as the evaluation of new targeted interventions, would benefit from the development of objective, standardized FOG measures capable of monitoring this debilitating symptom in a community setting.
We have previously described a technique for identification of freezing episodes based on the frequency characteristics of vertical leg acceleration . The algorithm is predicated on the observation that FOG entails an increase in high frequency (3-8 Hz) leg movement ('trembling') in the relative absence of lower frequency (0-3 Hz) locomotor activity, and the ratio of power in this 'freeze' and' locomotor' band has proven a strong marker of FOG . The original study was based on a small PD patient cohort (N=11) and utilized a single accelerometer on the left shank. In the current study we extend this technique to evaluate the effects of sensor location (back, thighs, shanks, feet) and signal processing parameters (sampling window width and the 'freeze' threshold) in a larger cohort of 'freezers' (N=25) to determine the optimal configurations for autonomous FOG identification from lower body accelerometry.
Twenty five patients (17 male, 8 female) who were attending the Parkinson’s Disease Research Clinic at the Brain and Mind Research Institute were identified for this study by self-reported freezing behavior. All patients satisfied UKPDS Brain Bank criteria , had a Mini-Mental State Examination (MMSE)  score of ≥ 24 and were deemed unlikely to have dementia or major depression according to DSM-IV criteria by consensus rating of a neurologist (SJGL) and a neuropsychologist (SLN). The study was approved by The University of Sydney Human Research and Ethics Committee and written informed consent was obtained.
Clinical evaluation and questionnaires
Patients were assessed in the clinically-defined ‘off’ state following overnight withdrawal of dopaminergic therapy. Six patients also had Deep Brain Stimulation (five participants with electrodes implanted in the subthalamic nucleus and one subject with electrodes in the pedunculopontine nucleus), which was turned off 60-min prior to assessment. Patient characteristics were as follows; mean age 69.0 [SD 8.4], disease duration 10.5 years [SD 8.2], Hoehn and Yahr stage  2.7 [SD 0.5], UPDRS – Section III 41.3 [SD 12.3], Parkinson's Disease Questionnaire (PDQ) 39  67.4 [SD 23.5], and NFOG-Q 18.2 [SD 5.9]. None of the patients described any increase in freezing behavior following the administration of their usual dopaminergic therapy.
Objective assessment of FOG
We have previously described a technique for objective identification of freezing episodes based on the frequency characteristics of vertical shank acceleration from a single sensor , which is summarized here. Peaks at the stride and step frequencies (approximately 1 and 2 Hz, respectively ) characterize the power spectrum of vertical leg acceleration during an epoch of walking, with the vast majority of power occurring at frequencies below 3 Hz . Thus 0-3 Hz was defined as the ‘locomotor’ band. In contrast, FOG is characterized by an absence of forward gait and the development of high frequency trembling of the lower limbs, which were found to occur primarily within 3 to 8 Hz ; this region was designated the ‘freeze’ band. These characteristic frequency signatures were used to discriminate FOG (during which power is significantly decreased or absent in the locomotor band relative to a large increase in power in the ‘freeze’ band) and normal gait (in which the vast majority of power is in the locomotor band). Specifically, for a particular period of leg acceleration a threshold could be set for the ratio of freeze to locomotor band power above which a particular epoch was identified as a FOG event. This freeze threshold ranged from 2.3 to 4.4 in individual patients  (i.e., FOG was associated with freeze band power approximately 2 to 4 times the power in the locomotor band).
Another parameter of interest is the duration of the sampling epoch. In our original study  50% of FOG episodes were found to be less than 10 s in duration, and 15% were under 4 s. Applying the FOG identification algorithm with window sizes >10 s tended to average out short FOG events, effectively acting as a low pass filter . Thus, for this study 10 s was considered the upper limit of sampling window size. A smaller window may be more sensitive to shorter FOG events, whereas a wider window of 10 s may improve specificity. The aim of the current study was to systematically explore the effects of freeze threshold and window size, as well as the use of multi-segmental sensor placement, on the sensitivity and specificity of FOG detection.
The results of autonomous FOG identification from combinations of sensors were compared in post-hoc analysis. A majority vote algorithm was used to identify FOG using all seven IMUs (back, thighs, shanks and feet); at each point in time the combined binary FOG waveform was set to '1' if 4 or more sensors indicated a freeze event (and '0' if less than 4 sensors registered FOG). FOG identification from each limb segment (thighs, shanks, feet) utilized a logical ‘OR’ operation; if the iFOG ratio exceeded the freeze threshold in either the left or right sensor (or occurred in both sensors) a freeze event was flagged. The ability of the single lumbar sensor to discriminate FOG was also evaluated. In addition, FOG identification from individual shank IMUs (left or right) was assessed to allow comparison with our previous study , which utilized a single sensor on the left shank. (Freeze identification from individual thigh and feet sensors were also assessed, but are not discussed in the manuscript to reduce complexity; for completeness, results from individual thigh and foot sensors are included as Additional file 1).
Clinical assessment of FOG
The frequency and relative duration of freezing episodes were determined for each video utilizing a FOG tagging program described previously [10, 24]. Two clinicians experienced in FOG (JMS and SJGL) used their best clinical judgment to identify FOG episodes from video, tagging the onset of a freeze by pressing the ‘T’ key and holding down the key for the duration of each event (Figure 1). Video editing tools enabled the ends of a horizontal bar representing the duration of each tagged freeze to be dragged backwards or forwards in time to facilitate accurate logging of FOG onset and offset. Analogous to the accelerometer processing, clinical ratings were saved as a binary signal at the video frame rate (30 Hz), with a zero baseline and 1 indicating freezing of gait. The results from the two raters were combined using a logical OR operation, resulting in a single binary FOG waveform for each TUG task. The number of FOG events and the percent time frozen were calculated from this signal (as described above), resulting in two arrays (number of FOG; percent time frozen) of 134 elements (corresponding to the 134 TUG trials). This rater pairing achieved high agreement (intraclass correlation coefficient [ICC] for number of FOG=0.82; percent time frozen ICC=0.99) in a previous study using this scoring technique .
The reliability of the quantification of the number of FOG episodes and percent time frozen was calculated between the clinical observers and the objective accelerometer-derived (iFOG) measures using the ICC ; the 134-element arrays for frequency, and relative duration of FOG, obtained from clinical assessment were compared to each of the 392 arrays representing the full range of window size (n=4), iFOG thresholds (n=14) and sensor locations (n=7). We used the following classification of ICC power: <0.2 negligible, 0.2≤0.4 weak, 0.4≤0.7 moderate, 0.7≤0.9 strong, and >0.9 very strong . Sensitivity-specificity analysis assessed the performance of each sensor group as a binary classification test, assuming that the subjective clinical ratings were a true 'gold standard'. For each of the 134-element arrays, sensitivity was defined as the ratio of the number of trials having at least one FOG event as determined from objective acceleration data over the number of trials having at least one FOG event as identified by the clinical raters. Similarly, specificity was defined as the ratio of the number of trials having no FOG activity as determined from objective acceleration data over the number of trials having no FOG activity as identified by the clinical raters.
Twenty out of the twenty-five participants exhibited clinically identifiable FOG during the study. A total of 298 FOG events (range 0-50 per subject; mean 11.9 [S.D. 13.4]) were identified from the video recordings by the clinical raters; percent time frozen averaged 24.1% [SD 24.8] (range 0-72.4%). The frequency and relative duration of FOG determined by the sensors was dependent on three parameters; 1) the sensor location(s), 2) the size of the sampling window, and 3) the iFOG threshold. For example, with a sampling window of 5s and an iFOG threshold of 3 (as shown in Figure 2) the results from all seven sensors (majority vote) were a total of 354 FOG events (range 0-59 per subject; mean 13.9 [S.D. 13.3]), and percent time frozen averaged 22.7% [SD 18.4] (range 0 - 66.3%).
Number of FOG
Percent time frozen
The agreement between clinicians and sensors for percent time frozen was essentially independent of the width of the sampling window (Figure 4E-H). The best performing locations were all seven IMUs (majority vote) and the individual left and right shank sensors, with ICCs>0.7 within a threshold range of 2-4.
Sensitivity and specificity
Recommended configurations for autonomous FOG identification from accelerometry with corresponding performance metrics
ICC number of FOG
ICC percent time frozen
shank (L or R)
The results from this study demonstrate the feasibility of objective monitoring of FOG from lower body accelerometry, and provide a practical basis for implementation based on the particular goal of the application. Three parameters must be considered; the location and number of sensors, the width of the sampling window for determining the ratio of freeze (3-8 Hz) to locomotor (0-3 Hz) band power at each point in time, and the threshold above which this ratio indicates a FOG event. These choices represent tradeoffs between sensitivity and specificity; the more distal the sensor, the narrower the window, and the lower the threshold, the higher the sensitivity of FOG detection at the expense of specificity (more false positives).
The use of seven sensors (back, thighs, shanks and feet) and a majority vote (at least 4 sensors registering FOG) provided the most consistently robust measure of freezing across the range of window and threshold values, and should be considered the 'reference' system against which other combinations of sensors are judged. A close second was the use of individual left or right shank accelerometers, which suggests that the 'trembling' associated with FOG is a bilateral ‘bottom-up’ phenomenon (higher power distally – see also Figure 2). A threshold of 3 was found to produce strong agreement with the clinical raters (ICC>0.7) for individual shank sensors at window sizes 5 s and above. This is consistent with our original study , in which the optimal individual threshold for a sensor on the left shank ranged from 2.3 to 4.4 at a window size of 6 s. At low FOG thresholds (<3) the ICC for percent time frozen was higher for individual shank sensors than the combined shanks (Figure 4). This was likely due to increased false positives in individual shank sensors when the threshold was set too low; logically combining these FOG measures lowered the agreement with the clinical observers. For iFOG thresholds ≥ 3 the combined shank ICC was consistent with single shank sensors. The single lumbar sensor provided sensitivity and specificity over 80% with a threshold of 3 and a window size of 10 s; although the high frequency lower-limb 'trembling' of FOG was attenuated by the musculoskeletal system relative to the more distal sensors (Figure 2A,D), sufficient power was present to reliably identify the presence of FOG.
Objective identification of the number of FOG events tended towards overestimation relative to the clinical observers; for example, using seven sensors with a 5 s second window and an iFOG threshold of 3 identified 354 discrete FOG events from the 134 TUG trials, 19% higher than the 298 episodes observed by the clinicians. This may be due to the ability of accelerometry to detect subclinical variations in trembling intensity not visible to the naked eye, resulting in a higher FOG count. Moreover, our recent work [10, 24] has demonstrated that percent time frozen is a more robust clinical metric of FOG severity than simply counting the number of freeze events, as it minimizes individual differences in scoring multiple sequential, or one longer, FOG event(s). Agreement (ICC) between 10 movement disorder specialists scoring a set of 14 videos of patients performing the TUG task was 16% higher for percent time frozen relative to number of FOG  (our two raters demonstrated a similar relative agreement in the same study, with an ICC of 0.82 for number of FOG and 0.99 for percent time frozen). In the current study, mean percent time frozen (all seven sensors; 5s window, iFOG threshold 3) per subject from accelerometry (22.7%) and clinical observation (24.1%) was in close agreement.
The results of this study demonstrate that objective FOG identification based on the frequency characteristics of lower body motion can achieve strong agreement (ICC > 0.7) with clinical assessment by movement disorder specialists. Advances in motion capture hardware may reduce the size and complexity of multi-segmental accelerometry, facilitating use in the community. Increasing the agreement between objective and subjective measures may require more than technical enhancement, given the current lack of standardized criteria for determining FOG onset and offset . The accelerometer-based approach provides the advantage of an adjustable, objective criteria for FOG onset and offset based on the frequency characteristics of lower-limb tremor; this may prove of benefit when training movement disorder specialists in clinical assessment of freezing, providing feedback on the nature of trembling associated with FOG.
Evaluation of FOG in clinical practice is notoriously difficult due to its paroxysmal nature, and current subjective approaches lack utility. There is a clear need for the development of objective standardized measures for assessment of future pharmacological and non-pharmacological interventions, and the results presented herein provide a practical basis for the implementation of accelerometry-based FOG monitoring. The seven-sensor approach was both the most complex and most robust system, scoring highly on all measures of performance, and is better suited to demanding research applications. A single shank sensor provided FOG measures comparable to the seven-sensor approach but is considerably simpler in execution, facilitating clinical use. A single lumbar sensor exhibited only moderate agreement with the clinical raters in terms of absolute number and duration of FOG events but had high sensitivity and specificity as a binary test of FOG, and lends itself readily to ambulatory monitoring given the ubiquitous nature of accelerometers in mobile telephones and other belt-worn devices.
The authors gratefully acknowledge the patients who participated in this study. Dr. Moore is supported by NASA grants NNX09AL14G and NNX12AM25G and the National Space Biomedical Research Institute; Dr. MacDougall is supported by a grant from the Garnett Passe and Rodney Williams Memorial Foundation; Dr. Naismith is supported by an NHMRC Career Development Award No. 1008117; Dr. Lewis is supported by an NHMRC Practitioner Fellowship and a University of Sydney Rolf Edgar Lake Postdoctoral Fellowship. There are no financial disclosures for Drs. Shine, Morris or Dilda to make.
- Bloem BR, Hausdorff JM, Visser JE, Giladi N: Falls and freezing of gait in Parkinson's disease: a review of two interconnected, episodic phenomena. Mov Disord 2004, 19:871–884.PubMedView Article
- Giladi N, Treves TA, Simon ES, Shabtai H, Orlov Y, Kandinov B, Paleacu D, Korczyn AD: Freezing of gait in patients with advanced Parkinson's disease. J Neural Transm 2001, 108:53–61.PubMedView Article
- Giladi N, McMahon D, Przedborski S, Flaster E, Guillory S, Kostic V, Fahn S: Motor blocks in Parkinson's disease. Neurology 1992, 42:333–339.PubMedView Article
- Macht M, Kaussner Y, Moller JC, Stiasny-Kolster K, Eggert KM, Kruger HP, Ellgring H: Predictors of freezing in Parkinson's disease: a survey of 6,620 patients. Mov Disord 2007, 22:953–956.PubMedView Article
- Fahn S, Elton RL: Unified Parkinson's disease rating scale. In Recent developments in Parkinson's disease. Volume 2. Edited by: Fahn S, Marsden CD, Goldstein M, Calne DB. Florham park, NJ: Macmillan healthcare Information; 1987:153–163. members. Up
- Giladi N, McDermott MP, Fahn S, Przedborski S, Jankovic J, Stern M, Tanner C: Freezing of gait in PD: prospective assessment in the DATATOP cohort. Neurology 2001, 56:1712–1721.PubMedView Article
- Giladi N, Tal J, Azulay T, Rascol O, Brooks DJ, Melamed E, Oertel W, Poewe WH, Stocchi F, Tolosa E: Validation of the freezing of gait questionnaire in patients with Parkinson's disease. Mov Disord 2009, 24:655–661.PubMedView Article
- Nieuwboer A, Rochester L, Herman T, Vandenberghe W, Emil GE, Thomaes T, Giladi N: Reliability of the new freezing of gait questionnaire: agreement between patients with Parkinson's disease and their carers. Gait Posture 2009, 30:459–463.PubMedView Article
- Rascol O, Brooks DJ, Melamed E, Oertel W, Poewe W, Stocchi F, Tolosa E: Rasagiline as an adjunct to levodopa in patients with Parkinson's disease and motor fluctuations (LARGO, Lasting effect in Adjunct therapy with Rasagiline Given Once daily, study): a randomised, double-blind, parallel-group trial. Lancet 2005, 365:947–954.PubMedView Article
- Shine JM, Moore ST, Bolitho SJ, Morris TR, Dilda V, Naismith SL, Lewis SJG: Assessing the utility of Freezing of Gait questionnaires in Parkinson’s Disease. Parkinsonism Relat Disord 2012, 18:25–29.PubMedView Article
- Hausdorff JM, Balash Y, Giladi N: Time series analysis of leg movements during freezing of gait in Parkinson's disease: rhyme or reason? Physica A 2003, 321:565–570.View Article
- Jacobs J, Nutt J, Carlsonkuhta P, Stephens M, Horak F: Knee trembling during freezing of gait represents multiple anticipatory postural adjustments. Exp Neurol 2009, 215:334–341.PubMedView Article
- Lo AC, Chang VC, Gianfrancesco MA, Friedman JH, Patterson TS, Benedicto DF: Reduction of freezing of gait in Parkinson's disease by repetitive robot-assisted treadmill training: a pilot study. J Neuroeng Rehabil 2010, 7:51.PubMedView Article
- Moore ST, MacDougall HG, Ondo WG: Ambulatory monitoring of freezing of gait in Parkinson's disease. J Neurosci Meth 2008, 167:340–348.View Article
- Snijders AH, Weerdesteyn V, Hagen YJ, Duysens J, Giladi N, Bloem BR: Obstacle avoidance to elicit freezing of gait during treadmill walking. Mov Disord 2010, 25:57–63.PubMedView Article
- Chee R, Murphy A, Danoudis M, Georgiou-Karistianis N, Iansek R: Gait freezing in Parkinson's disease and the stride length sequence effect interaction. Brain 2009, 132:2151–2160.PubMedView Article
- Moreau C, Defebvre L, Bleuse S, Blatt JL, Duhamel A, Bloem BR, Destée A, Krystkowiak P: Externally provoked freezing of gait in open runways in advanced Parkinson’s disease results from motor and mental collapse. J Neural Transm 2008, 115:1431–1436.PubMedView Article
- Nieuwboer A, Chavret F, Willems A-M, Desloovere K: Does freezing in Parkinson’s disease change limb coordination? J Neurol 2007, 254:1268–1277.View Article
- Nieuwboer A, Dom R, De Weerdt W, Desloovere K, Janssens L, Stijn V: Electromyographic profiles of gait prior to onset of freezing episodes in patients with Parkinson's disease. Brain 2004, 127:1650–1660.PubMedView Article
- Bartels AL, Balash Y, Gurevich T, Schaafsma JD, Hausdorff JM, Giladi N: Relationship between freezing of gait (FOG) and other features of Parkinson's: FOG is not correlated with bradykinesia. J Clin Neurosci 2003, 10:584–588.PubMedView Article
- Hausdorff JM, Schaafsma JD, Balash Y, Bartels AL, Gurevich T, Giladi N: Impaired regulation of stride variability in Parkinson's disease subjects with freezing of gait. Exp Brain Res 2003, 149:187–194.PubMed
- Schaafsma JD, Balash Y, Gurevich T, Bartels AL, Hausdorff JM, Giladi N: Characterization of freezing of gait subtypes and the response of each to levodopa in Parkinson's disease. Eur J Neurol 2003, 10:391–398.PubMedView Article
- Ziegler K, Schroeteler F, Ceballos-Baumann AO, Fietzek UM: A new rating instrument to assess festination and freezing gait in Parkinsonian patients. Mov Disord 2010, 25:1012–1018.PubMedView Article
- Morris TR, Cho C, Dilda V, Shine JM, Naismith SL, Lewis SJ, Moore ST: A comparison of clinical and objective measures of freezing of gait in Parkinson's disease. Parkinsonism Relat Disord 2012, 18:572–577.PubMedView Article
- Schrag A, Quinn N: Dyskinesias and motor fluctuations in Parkinson's disease: A community based study. Brain 2000, 123:2297–2305.PubMedView Article
- Stefani A, Lozano AM, Peppe A, Stanzione P, Galati S, Tropepi D: Bilateral deep brain stimulation of the pedunculopontine and subthalamic nuclei in severe Parkinson's disease. Brain 2007, 130:1596–1607.PubMedView Article
- Gibb WR, Lees AJ: A comparison of clinical and pathological features of young- and old-onset Parkinson's disease. Neurology 1988, 38:1402–1406.PubMedView Article
- Folstein MF, Folstein SE, McHugh PR: Mini-mental state. A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res 1975, 12:189–198.PubMedView Article
- Hoehn MM, Yahr MD: Parkinsonism: onset, progression and mortality. Neurology 1967, 17:427–442.PubMedView Article
- Jenkinson C, Fitzpatrick R, Peto V, Greenhall R, Hyman N: The Parkinson's Disease Questionnaire (PDQ-39): development and validation of a Parkinson's disease summary index score. Age Ageing 1997, 26:353–357.PubMedView Article
- MacDougall HG, Moore ST: Marching to the beat of the same drummer: the spontaneous tempo of human locomotion. J Appl Physiol 2005, 99:1164–1173.PubMedView Article
- Shrout PE, Fleiss JL: Intraclass correlations: uses in assessing rater reliability. Psychol Bull 1979, 86:420–428.PubMedView Article
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.