Measurement of errors in quantitative gait data:
Errors, Reliability and Repeatability

• A comparison of hip joint centre localisation techniques with 3-DUS
A comparison of hip joint centre localisation techniques with 3-DUS for clinical gait analysis in children with cerebral palsy

"Functional calibration techniques have been proposed as an alternative to regression equations for estimating the position of the hip within the pelvic co-ordinate system for clinical gait analysis. So far validation of such techniques has focussed on healthy adults. This study evaluated a range of techniques based on regression equations or functional calibration procedures techniques in 46 children representative of those attending a major clinical gait analysis service against previously validated 3-D ultrasound techniques for determining the hip joint centre."

-----

Abstract:

• Analysis of measurement performance in optical motion capturing
"Validity and reliability as scientific quality criteria have to be considered when using motion capture systems for research purposes. System characteristics such as accuracy and precision are often not addressed in scientific reports of human motion analysis studies, even though literature and standards recommend individual laboratory setup evaluation."

Paper: Journal of Biomechanics 49 (2016) 2085–2088

• Analysis of accuracy in optical motion capture
Analysis of accuracy in optical motion capture - A protocol for laboratory setup evaluation

"Validity and reliability as scientific quality criteria have to be considered when using optical motion capture (OMC) for research purposes. Literature and standards recommend individual laboratory setup evaluation. However, system characteristics such as trueness, precision and uncertainty are often not addressed in scientific reports on 3D human movement analysis. One reason may be the lack of simple and practical methods for evaluating accuracy parameters of OMC.

A protocol was developed for investigating the accuracy of an OMC system (Vicon, volume 5.5×1.2×2.0m³) with standard laboratory equipment and by means of trueness and uncertainty of marker distances. The study investigated the effects of number of cameras (6, 8 and 10), measurement height (foot, knee and hip) and movement condition (static and dynamic) on accuracy. Number of cameras, height and movement condition affected system accuracy significantly. For lower body assessment during level walking, the most favorable setting (10 cameras, foot region) revealed mean trueness and uncertainty to be
-0.08 and 0.33mm, respectively."
-----


• Assessment of the kinematic variability among 12 motion analysis laboratories
"Variability of kinematic measurements among sites participating in a collaborative research investigation is a primary factor in determining number of subjects, level of detectable difference and statistical power of a multi-site research study. In this study, one subject was evaluated by 24 examiners at 12 motion analysis laboratories and the observed variability of nine kinematic parameters are reported."
• Correcting lower limb segment axis misalignment
Correcting lower limb segment axis misalignment in gait analysis: A simple geometrical method

"Research question: This study proposes an a posteriori geometrical method based solely on segment anatomy that aims to correct the thigh medial-lateral axis definition and crosstalk-related error.

Methods: The proposed method considers the thigh medial-lateral axis as the normal to the mean sagittal plane of the lower limb defined by hip, knee and ankle joint centres during one gait cycle. Its performance was compared to that of an optimisation method which repositions the axis to reduce knee abduction-adduction variance."


• Effects of hip joint centre mislocation on gait kinematics
Effects of hip joint centre mislocation on gait kinematics of children with cerebral palsy calculated using patient-specific direct and inverse kinematic models

"Joint kinematics can be calculated by Direct Kinematics (DK), which is used in most clinical gait laboratories, or Inverse Kinematics (IK), which is mainly used for musculoskeletal research. In both approaches, joint centre locations are required to compute joint angles. The hip joint centre (HJC) in DK models can be estimated using predictive or functional methods, while in IK models can be obtained by scaling generic models. The aim of the current study was to systematically investigate the impact of HJC location errors on lower limb joint kinematics of a clinical population using DK and IK approaches."


• Intra- and Interobserver Reliability Comparison of CGA Data (open access)
Intra- and Interobserver Reliability Comparison of Clinical Gait Analysis Data between
Two Gait Laboratories

"Abstract: Comparing clinical gait analysis (CGA) data between clinical centers is critical in the treatment and rehabilitation progress. However, CGA protocols and system configurations, as well as choice of marker sets and individual variability during marker attachment, may affect the comparability of data. The aim of this study was to evaluate reliability of CGA data collected between two gait analysis laboratories. Three healthy subjects underwent a standardized CGA protocol at two separate centers. Kinematic data were captured using the same motion capturing systems (two systems, same manufacturer, but diffrent analysis software and camera configurations). The CGA data were analyzed by the same two observers for both centers."


• Measurement and management of errors in quantitative gait data
"Gait analysis is a valuable tool in the evaluation of children and adults with movement disorders. The data produced from gait analysis, however, is not necessarily free of errors. The purpose of this study was two-fold: (i) to estimate the errors associated with quantitative gait data; and (ii) to propose a method for incorporating the knowledge of these errors into the clinical interpretation process."

-----

Abstract:

• Measurement of the screw-home motion of the knee is sensitive to errors in axis alignment
"Several authors have reported on the "screwhome" motion of the knee joint (external rotation of the tibia with respect to the femur during extension) with varying results (Kurosawa et al., 1985; Lafortune et al., 1992). These discrepancies in measurements of axial knee rotation may be due to errors caused by kinematic "crosstalk". Such errors arise when the chosen knee joint coordinate system is not aligned with anatomical axes (e.g., when the flexion-extension axis is not aligned in the mediolateral direction).

The purpose of the present study is to investigate the possibility that screw-home motion of the knee is a manifestation of kinematic crosstalk. It may be that external rotation of the tibia does not accompany knee extension but rather is knee extension that appears as rotation about a different axis. We hypothesize that small misalignments of the knee joint coordinate system may result in knee extension being misinterpreted as external rotation of a magnitude consistent with reports of measured screw-home motion. Specifically, we will test (1) whether a screw-home motion can be measured where none exists, and (2) whether kinematic crosstalk can hide a screw-home motion that is known to occur."

Paper: Journal of Biomechanics 33 (2000) 1029-1034

• Measurement of the screw-home motion of the knee is sensitive to errors in axis alignment
"Measurements of joint angles during motion analysis are subject to error caused by kinematic crosstalk, that is, one joint rotation (e.g., flexion) being interpreted as another (e.g., abduction).Kinematic crosstalk results from the chosen joint coordinate system being misaligned with the axes about which rotations are assumed to occur.The aim of this paper is to demonstrate that measurement of the so-called "screw-home" a motion of the human knee, in which axial rotation and extension are coupled, is especially prone to errors due to crosstalk."
-----


• Quantitative comparison of five current protocols in gait analysis
"Data collection and reduction procedures, coherently structured in protocols, are necessary in gait analysis to make kinematic and kinetic measurements clinically comprehensible. The current protocols differ considerably for the marker-set and for the biomechanical model implemented. Nevertheless, conventional gait variables are compared without full awareness of these differences.
A comparison was made of five worldwide representative protocols by analysing kinematics and kinetics of the trunk, pelvis and lower limbs exactly over the same gait cycles. A single comprehensive arrangement of markers was defined by merging the corresponding five marker-sets. This resulted in 60 markers to be positioned either on the skin or on wands, and in 16 anatomical landmark calibrations to be performed with an instrumented pointer. Two healthy subjects and one patient who had a special two degrees of freedom knee prosthesis implanted were analysed. Data from up-right posture and at least three gait repetitions were collected. Five corresponding experts participated in the data collection and analysed independently the data according to their own procedures."
• Quantifying sources of variability in gait analysis
"Measurements from gait analysis are affected by many sources of variability. Schwartz et al. illustrated an experimental design and methods to estimate these variance components. However, the derivation contains errors which could severely bias the estimation of some components. Therefore, in this paper, we presented correction to this method using ANOVA and Likelihood methods."
• Reliability of four models for clinical gait analysis
"Three-dimensional gait analysis (3DGA) has become a common clinical tool for treatment planning in children with cerebral palsy (CP). Many clinical gait laboratories use the conventional gait analysis model (e.g. Plug-in-Gait model), which uses Direct Kinematics (DK) for joint kinematic calculations, whereas, musculoskeletal models, mainly used for research, use Inverse Kinematics (IK). Musculoskeletal IK models have the advantage of enabling additional analyses which might improve the clinical decision-making in children with CP. Before any new model can be used in a clinical setting, its reliability has to be evaluated and compared to a commonly used clinical gait model (e.g. Plug-in-Gait model) which was the purpose of this study. Two testers performed 3DGA in eleven CP and seven typically developing participants on two occasions. Intra- and inter-tester standard deviations (SD) and standard error of measurement (SEM) were used to compare the reliability of two DK models (Plug-in-Gait and a six degrees-of-freedom model solved using Vicon software) and two IK models (two modifications of 'gait2392' solved using OpenSim). All models showed good reliability (mean SEM of 3.0° over all analysed models and joint angles)."
• Repeatability of gait data using a functional hip joint centre and a mean helical knee axis
"The purpose of this paper was to compare the repeatability of gait data obtained from two models, one based on ALs (AL model), and the other incorporating a functional method to define hip joint centres and a mean helical axis to define knee joint flexion/extension axes (FUN model)."
• Statisticians issue warning on P values
"Misuse of the P value — a common test for judging the strength of scientific evidence — is contributing to the number of research findings that cannot be reproduced, the American Statistical Association (ASA) warned on 8 March. The group has taken the unusual step of issuing principles to guide use of the P value, which it says cannot determine whether a hypothesis is true or whether results are important."
• The clinical impact of hip joint centre regression equation error
The clinical impact of hip joint centre regression equation error on kinematics and kinetics during paediatric gait

"Regression equations based on pelvic anatomy are routinely used to estimate the hip joint centre during gait analysis. While the associated errors have been well documented, the clinical significance of these errors has not been reported. This study investigated the clinical agreement of three commonly used regression equation sets (Bell et al., Davis et al. and Orthotrak software) against the equations of Harrington et al."


• The effect of subject measurement error on joint kinematics in the conventional gait model
The effect of subject measurement error on joint kinematics in the conventional gait model:
Insights from the open-source pyCGM tool using high performance computing methods

"The conventional gait model (CGM) is a widely used biomechanical model which has been validated over many years. The CGM relies on retro-reflective markers placed along anatomical landmarks, a static calibration pose, and subject measurements as inputs for joint angle calculations. While past literature has shown the possible errors caused by improper marker placement, studies on the effects of inaccurate subject measurements are lacking. Moreover, as many laboratories rely on the commercial version of the CGM, released as the Plug-in Gait (Vicon Motion Systems Ltd, Oxford, UK), integrating improvements into the CGM code is not easily accomplished. This paper introduces a Python implementation for the CGM, referred to as pyCGM, which is an open-source, easily modifiable, cross platform, and high performance computational implementation. The aims of pyCGM are to (1) reproduce joint kinematic outputs from the Vicon CGM and (2) be implemented in a parallel approach to allow integration on a high performance computer. The aims of this paper are to (1) demonstrate that pyCGM can systematically and efficiently examine the effect of subject measurements on joint angles and (2) be updated to include new calculation methods suggested in the literature."

Project: python Convention Gait Model (pyCGM)


• The reliability of three-dimensional kinematic gait measurements: A systematic review
"Three-dimensional kinematic measures of gait are routinely used in clinical gait analysis and provide a key outcome measure for gait research and clinical practice. This systematic review identifies and evaluates current evidence for the inter-session and inter-assessor reliability of threedimensional kinematic gait analysis (3DGA) data."
• Validation of hip joint center localization methods during gait analysis
Validation of hip joint center localization methods during gait analysis using 3D EOS imaging in typically developing and cerebral palsy children

"Localization of the hip joint center (HJC) is essential in computation of gait data. EOS low dose biplanar X-rays have been shown to be a good reference in evaluating various methods of HJC localization in adults. The aim is to evaluate predictive and functional techniques for HJC localization in typically developing (TD) and cerebral palsy (CP) children, using EOS as an image based reference."

Biomech-L forum:

• 3DMA reliability (03-22-2017), Dr. Allan Carman
"This post relates to Chris Kirtley's comment "…it seems to me the field has not progressed much in that last 10 years" as well as a comment on poor reliability and clinical 3D gait analysis (Thread: “Best web-site for interaction with doctors engaged with human gait disorder”). It also follows on from my previous posts which presented and understanding of axes-misalignment, 3DMA methods and normative gait data. The later highlighted to me the lack of awareness of the reliability of different 3DMA methods. To put some perspective on the comparative reliability of different methods and the progress made over the last 20+ years in 3DMA I have presented the results of a systemic review from a 4th year student research project.

Methods include traditional anatomical land mark based, KAD, Optimization methods and rigid fixation devices with and without functional joint centre methods. A summary Excel spread sheet is attached and can be found at:

Google drive folder: 3DMA Reliability or excel file: ResultsTableSytematic Review brief.xlsx

Google drive preview does not show any plots.

Key points:

1-Intra-session (same marker placement) reliability for the majority of studies was very poor and unacceptable for non-sagital rotations of the hip, knee and ankle. Acceptable intra-session reliability has been produced for a number of studies for a few joint angles, irrespective of type of method, but this is the exception.
2-Large variability is seen within each type of study design across intra-session, inter session and inter-examiner results. Therefore no design can be considered reliable and is highly dependent on the implementation of the method.

There is something fundamentally wrong with the processes undertaken to derive joint angle data for the majority of reliability studies. So much so that often the gait variability within session is far larger than the total variability expected between subjects during normal gait.

Comparison of methods:

1-Traditional marker based anatomical landmark (AL) based methods of the 80’s and 90’s (Kit Vaughan, mod-HH, VCM, PiG) were known to suffer from cross-talk (axes misalignment and non-linear errors) and poor reliability in the non-sagital knee joint angle data.
2-The KAD was introduced (90’s ?) to help define thigh medio-lateral (knee flex/ext) axis alignment to improve reliability and reduce cross talk. However it had limited success with marginal improvements in reliability in gait kinematics over the traditional marker based methods. The influence of the examiner’s interpretation and implementation of the method and variability in KAD placement was still a limiting factor.
3-Optimisation approaches of the 2000’s saw a variety of methods to align medio-lateral knee axes based on minimizing knee abd/add cross talk. Including PCA or best fit to an ideal knee abd/add curve during gait. Although only a few studied and varying in their results between approaches, these methods saw a significant improvement in reliability over KAD or traditional AL methods.

From here the methods have gone backwards.

1-The T3DGait (2007) saw a return to the anatomical landmarks and minimal marker numbers of the traditional AL methods.
2-Function methods for joint centres (2010’s) have also seen a return, but have produced no improvement in reliability over traditional AL methods, adding time and complexity to the analysis for no benefit. Although both KAD and function methods are unreliable, it could argued that for inter-session studies you should stick with the KAD method as it has a more controlled approach producing less variability between reliability studies. The function method lacks a clearly defined protocol in terms of planes of movement, RoM, or controlling pelvic motion, potentially leading to larger variability in outcome measures between reliability studies.
3-A resent addition is the IK model for gait analysis. Although there are no reliability studies, published comparative gait data, and knowledge of factors that affect reliability and validity, indicate that this is possibly the worst method presented for 3D gait analysis. Combining the worst in 3DMA including whole body global optimization, 1 df knee joint, RFD on the thigh and functional joint centers.

These later methods have all ignored previous literature and work on axes misalignment and non-linear error (cross talk), with no attempt to assess or correct for axes mis-alignment.

Something that has been long missing in the 3DMA literature is criteria against which to measure reliability outcome measures. This is part of the reason why every reliability study that I have read concludes that their methods are reliable and suitable for 3DMA gait analysis, when clearly they are not. This conclusion has been based on a vaguely chosen value (such as ICC or CMC >0.8) that has no relevance to gait curves and is applied to all joint angle data despite it being known that the ROM varies considerably between different joint df’s (from 6 to 60 degs.).

I will end with an important consideration on the progress of 3DMA and a future trend. There was a huge leap in 3DMA technology in the mid 90’s which saw the emergence of the multi-camera, 3D tracking, real-time gait technology we see today. The 3DMA methods (marker sets, protocols etc.) have been left behind and have in recent years (in my opinion) gone backwards through a lack of critical thought, understanding and research into the underlying methods. There is currently another boom happening with wireless inertial sensors and their application to 3DMA. These come with claims of accuracy and reliability and are being promoted as tools for research and clinical gait analysis. In light of what I have presented in comparative reliability as well as previous posts on marker based methods, understanding of axes misalignment and normative gait data, these claims should be taken with caution. This is where I feel that 3DMA societies/groups and those conducting fundamental research into 3DMA methods need to step up to provide leadership in an area of 3DMA that is again changing rapidly with the introduction of new technology. If as Paul Devita says it wants to become the “breakthrough science of the 21st century”. To do this however requires a huge step forward in the science and current understanding of 3DMA methods, validity and reliability."