In this method, one person was considered as a test and the remaining persons were used as the training dataset. The average accuracy of the models is reported in Table 5.2. The RMSE and NRMSE of estimated lower body joint angles at different planes were less than 6.5 and 10 % for inter-participant models. An example of estimated angles compared with the angles measured by the motion capture system is shown in Figure 5.7. The accuracy of the inter-participant mod- els for different speeds is also shown in Figure 5.8 and Table A2 (Appendix A). In this section models are the same models trained for inter-participant tests, but the test data were split into different speeds. Then, the accuracy and error of the model at different speeds of test data were measured.
Figure 5.5: An Example of angles measured by strain sensors in intra-participant tests versus an- gles measured by motion capture system in a)hip sagittal plane b) hip frontal plane c)hip trans- verse plane d) knee sagittal plane e) ankle sagittal plane f ) ankle frontal plane.
7 8 9 10 11 12 13 0.75 0.8 0.85 0.9 0.95 1 Speed(km/h) R 2 Hip Sagittal Hip Frontal Hip Transverse Knee Sagittal Ankle Sagittal Ankle Frontal
Ankle Frontal for Excluded Anomaly Trial
Figure 5.6: Comparison of the R2 error of the estimated angle at different speeds in the intra- participant evaluation. The performance of the sensor and algorithm were consistent at different speeds.
Table 5.2: The average accuracy of inter-participant CNN models among 10 participants.
Hip Sagittal Knee Sagittal Ankle Sagittal R2 0.85 (0.11) 0.93 (0.04) 0.81 (0.11)
RMSE(deg) 5.39 (2.29) 6.38 (2.32) 3.92 (0.73)
NRMSE(%) 9.34 (3.41) 6.30(1.90) 9.99 (2.92)
5.6 Discussion
In this work, we developed a fabric-based wearable sensor and used a deep convolutional neural network to estimate lower body kinematics in sagittal, frontal, and transverse planes. A fiber- based strain sensor coated with insulating sheath was used for prototyping [66].
Compared to the previous studies, we have expanded the lower body monitoring from only sagittal plane to include frontal and transverse planes during running. In the intra-participant evaluation, estimated values in the sagittal plane were more accurate compared with other planes. The lowest R2 of 0.97 was obtained for sagittal plane while the lowest R2of 0.88 was obtained for frontal and transverse planes for person-specific models. Figure 5.5 demonstrates that there is a greater difference between angles measured by strain sensors and motion capture in hip frontal, hip transverse, and ankle frontal planes. We reasoned the difference of accuracy in sagit- tal and non-sagittal planes was from the complexity of movements, smaller angles that require
Figure 5.7: An example of angles measured by strain sensors in inter-participant tests versus an- gles measured by motion capture system in a)hip sagittal plane b) knee sagittal plane c) ankle sagittal plane. Pattern and range of motion of ankle and hip were more different among partici- pants compared with knee.
detection, and the sensor/CNN’s ability to separate these smaller angle changes in frontal and transverse planes from the larger sagittal plane. Since the range of motion in the sagittal plane was larger than other planes, sensors were affected more with flexion-extension than abduction- adduction and rotation during running. Therefore, the sensor’s signal was highly correlated with sagittal angles and less correlated with non-sagittal angles. Figure 5.5 and Figure 5.7 show the gait pattern in frontal and transverse planes was more complex than the sagittal plane.
A previous study showed that errors in estimation increased at higher speeds [26]. In contrast, in this study, similar accuracy at fast and slow speeds was achieved(Figure 5.6 and Figure 5.8) thanks to the deep convolutional neural network and sensor characteristics [66]. According to the sensor characteristics—frequency and strain rate—the sensor was able to track at low– and high–frequency (up to 10 Hz, Figure A1 Appendix A) [66]. The CNN models can better predict nonlinear patterns (hysteresis and time-dependent behavior that are inevitable in piezoresistive sensors at higher frequencies and strain rates) compared to linear models used in the previous
7 8 9 10 11 12 13 0.75 0.8 0.85 0.9 0.95 1 Speed(km/h) R 2 Hip Sagittal Knee Sagittal Ankle Sagittal
Figure 5.8: Comparison of the R2 error of the estimated angle at different speeds in inter- participant evaluation. The performance of the sensor and algorithm were consistent at different speeds.
works and therefore, lower errors at higher speeds were obtained in contrast to previous work [26]. Figure 5.5 demonstrates a lower accuracy of ankle frontal tracking at the speed of 11 km/h compared to the other speeds, even at the higher speed of 12 km/h. The decrease in accuracy for this trial was due to an anomaly in the data set of one participant. The ankle frontal values changed such that it had a detrimental effect on the CNN model’s ability to predict this trial when it was assigned as the test set. It is worth noting that this was directly correlated to a lower range of motion during the 11km/h speed, and was not consistent with the participant’s ankle frontal range of motion in all other speed trials. This negatively affected the R2 value and for comparison, we have included an extra data point that indicates the average for ankle frontal with this data set removed, which is consistent with the other R2values for frontal and transverse values (Figure 5.6).
Validation of gold standard optical motion capture system in measuring actual angles in frontal and transverse planes between participants has been illusive thus far [79]. Therefore, in inter-participant evaluation, the model was trained to estimate only sagittal plane angles. The patterns of knee joint angles were more consistent among participants compared to the pattern of the ankle and hip joint angles, and smaller errors were obtained in estimating knee angles.
Figure 5.5 demonstrates that there is a greater difference between estimated and reference an- gles of the hip and ankle versus knee joint. The R2and NRMSE of the estimated knee angles were 0.93 and 6.30 %, respectively, while the ankle and hip angles were estimated with R2and NRMSE of 0.81 and 9.9 %, and 0.85 and 9.34 %, respectively. The CNN model’s accuracy was lower when the pattern of joint angles in the test dataset was different from the patterns of joint angles in the training dataset. It can be expected that having a large dataset that includes different gait patterns would lead to a smaller error in the ankle and hip joint angle estimations.
The results of this study had smaller errors compared with previous works. Mengüç et al. used hyper-elastic sensors for sagittal plane monitoring of runners [26] and could obtain RMSE less than 15◦, 10◦and 6◦ for knee, hip, and ankle, respectively, while we have achieved RMSE of 1.12◦, 2.2◦and 1.3◦for knee, hip, and ankle joint, respectively. Capacitive strain sensors were used to measure the multi-axis ankle angle and have obtained an RMSE of less than 4◦[45]; while in this study ankle joint in sagittal and frontal planes were measured with an RMSE of less than 1.56◦. The method used in this work, including sensor placement and the convolutional neural network employed for angle estimation, outperformed previous studies.
In this study, a CNN model was used for angle estimation. The advantage of CNN is the abil- ity to automatically extract features from the input signal by convolutional layers. However, CNN models work better when a large-scale dataset is available. There are opportunities for using the fabric-based wearable device proposed in this study and CNN models for other purposes such as gait phase detection, rehabilitation monitoring, and Parkinson’s disease monitoring accord- ing to previous works [39, 80]. The challenges of using wearable fabric-based sensors compared to other technologies such as IMU-based systems and camera-based systems are changes in the performance due to differences in body-shapes and sensor positions, garment drifts during us- age, and designing the system to be washable.
The limitations of this research include the diversity of the cohort, testing environment, and test lengths. Expansion of this to people of different sex, sizes, and ages would allow testing on the abilities of the CNN model to track accurately with limited vs. expansive training data sets. The testing was conducted in a laboratory setting using a treadmill to control the speed with short data recording times. Outdoor non-treadmill running could result in different gait patterns with
different terrain. Testing periods were limited and long–term testing would be useful to analyze the performance and fit of the garment over time. To enable this type of testing alternative data recording or wireless capabilities would be required.
This work has addressed the challenge of creating reliable and accurate soft-sensor wearable devices that can track kinematic motion comparable to motion capture devices. The sensor fab- rication for the intended working range (and ability to withstand sweat), device design and pro- duction, and implementation of CNN to create a complete device were challenges of this study. We have addressed the challenge of obtaining a reliable kinematic motion capturing device that uses deep learning architecture to generalize for inter-participant data beyond what is currently available.
Chapter 6
Conclusion
6.1 Achievements
There is a need for an unobtrusive system for lower body motion monitoring for applications such as correction and modification of runners’ gait, in-home rehabilitation monitoring, and soft robotics. In this thesis, we proposed addressing this demand by using a fiber-based strain sensor. Previous works demonstrated the performance of soft strain sensors for lower body tracking in simple applications or limited degrees of freedom [26, 45]. We extended the use of these sensors for three dimensional whole lower body monitoring. Moreover, the potential low accuracy of the system was targeted by using deep supervised learning methods. The major contribution of this thesis work towards the objective proposed in the first chapter were:
• Extending the use of strain sensors for lower body tracking from two-dimensional to three- dimensional.
• Obtaining an error of less than 2.5 degrees among all lower body joints and axes that out- performs previous studies.
Defining a model that overcomes non-linear behavior of the sensor is a challenging problem. Recently, recurrent neural network models were employed and showed promising performance to overcome sensor limitations [57]. In this work, we set up a test to investigate the performance of the sensor in long data recording under a random pattern movement. In contrast to previ- ous works [57], this test better simulates real scenario movements of sensors for human motion monitoring. Moreover, recording data for 8 hours reflected any effects of drift in sensor values to be addressed by the calibration algorithm.
Although estimating model plays an important role in the application of these sensors, to the best of our knowledge, there was not a comprehensive comparison of possible methods in the literature. In this work, different supervised learning methods including random forest, neu- ral networks, recurrent neural networks, LSTM, and convolutional neural networks were evalu- ated and compared for characterization of strain sensors. Comparing different machine learn- ing methods for sensor characterization and also real-time application, CNN and random forest demonstrated the best results. While CNN showed high accuracy and less variance between dif- ferent participants, the random forest also demonstrated an equally high accuracy.
The performance of the wearable sensors on walking and running tests outperformed the previous works on human motion monitoring using strain sensors [26, 45, 44]. In intra-participant tests, an RMSE of less than 2◦for walking, and less than 2.5◦were obtained for running. Com- paring the results of Chapter 3 and Chapter 5, it should be considered that the performance of the sensor itself has improved and a sensor with better performance was used in the later study [66]. Although the results achieved in the running and walking study outperformed previous works [26, 26, 45, 44] the platform still requires to be calibrated with each specific participant for very accurate measurements. Attempts were made in this study to eliminate calibration for new participants and an RMSE of less than 4.5◦ was achieved for walking and running using
inter-participant training. There are some inherent problems in estimating human motion by measuring the strain of garments including, changes in body shapes, changes in sensor posi- tions between sessions, slides of the garment during a test. Considering the above mentioned problems, there is a boundary on the accuracy of the system for inter-participant tests.
6.2 Limitations and Future Works
This work had the following limitations:
• Performance of the system was tested in a lab setting and during controlled movements. • This study only considered short data recording for running and walking, while increasing
data recording time can raise some challenges including slides of the garment on the skin and drift of the sensor’s signal.
• The prototype was connected to a DAQ by wires for both walking and running studies and data were gathered and processed offline. The prototype needs to be wireless for outdoor use and portable applications.
• Studies on walking, running, and sensor placement were conducted merely on male par- ticipants. Due to the inherent differences between male and female participants’ body shapes, separate studies should be conducted on female participants.
• In this study performance of the system for intra-participant tests was evaluated by train- ing a model for each specific individual. The test-retest, inter-session, and inter-day val- idation of the machine learning models should be investigated. This question should be answered whether participants need to calibrate the system every session or it can be cal- ibrated once. A potential decrease in accuracy for inter-session and inter-day tests should be investigated.
Future works should address the use of the proposed platform for out-door running. It has been reported that there are differences between indoor and out-door running gait. Generalization of the system for actual out-door running should be investigated. The final objective of this work would be to use the proposed wearable system for real-time feedback training. Implementation of the algorithm on a micro-controller and substituting the wires in the prototype with coated conductive threads would be essential for out-door use of the system.
This study demonstrated the accuracy of a wearable fabric-based system for lower body kine- matic monitoring. The future works should concentrate on how the data gathered by the proto- type should be interpreted to have clinical applications. For instance, it has been reported that symmetries of right and left leg kinematics during running and changes in kinematic during long-running tests are indications of injuries. The performance of the system to address these measure specific clinical factors should be investigated.
This study demonstrated that the system requires calibration for each specific individual to have an accurate kinematic measurement. This entails that every individual calibrates the sys- tem in a gait analysis lab in advance that is not convenient. Future works should concentrate
on the fusion of multiple sensors such as strain sensors and IMUs to eliminate the calibration process.
Bibliography
[1] Charles G. Helmick and Sylvia I. Watkins-Castillo. The Burden of Musculoskeletal Diseases in the United States. Online; accessed 2019-10-26.
[2] Bhushan R. Deshpande, Jeffrey N. Katz, Daniel H. Solomon, Edward H. Yelin, David J. Hunter, Stephen P. Messier, Lisa G. Suter, and Elena Losina. Number of Persons With Symp- tomatic Knee Osteoarthritis in the US: Impact of Race and Ethnicity, Age, Sex, and Obesity: Symptomatic Knee OA in the US. Arthritis Care & Research, 68(12):1743–1750, December 2016.
[3] Randall Dick, Julie Agel, and Stephen W Marshall. National Collegiate Athletic Association Injury Surveillance System commentaries: introduction and methods. Journal of athletic training, 42(2):173–182, 2007.
[4] R N van Gent, D Siem, M van Middelkoop, A G van Os, S M A Bierma-Zeinstra, and B W Koes. Incidence and determinants of lower extremity running injuries in long distance runners: a systematic review. British Journal of Sports Medicine, 41(8):469–480, 2007.
[5] Brian Noehren, Michael B Pohl, Zack Sanchez, Tom Cunningham, and Christian Latter- mann. Proximal and distal kinematics in female runners with patellofemoral pain. Clinical biomechanics (Bristol, Avon), 27(4):366–371, May 2012.
[6] Richard W Willy, Kurt T Manal, Erik E Witvrouw, and Irene S Davis. Are mechanics different between male and female runners with patellofemoral pain? Medicine and science in sports and exercise, 44(11):2165–2171, November 2012.
[7] Anthony G. Schache, Peter D. Blanch, David A. Rath, Tim V. Wrigley, and Kim L. Bennell. Are anthropometric and kinematic parameters of the lumbo-pelvic-hip complex related to running injuries? Research in Sports Medicine (Print), 13(2):127–147, June 2005.
[8] Jonathan P. Folland, Sam J. Allen, Matthew I. Black, Joseph C. Handsaker, and Stephanie E. Forrester. Running Technique is an Important Component of Running Economy and Per- formance. Medicine and Science in Sports and Exercise, 49(7):1412–1423, 2017.
[9] Christopher Bramah, Stephen J. Preece, Niamh Gill, and Lee Herrington. Is There a Patho- logical Gait Associated With Common Soft Tissue Running Injuries? The American Journal of Sports Medicine, 46(12):3023–3031, 2018.
[10] CRISTINE AGRESTA and ALLISON BROWN. Gait Retraining for Injured and Healthy Run- ners Using Augmented Feedback: A Systematic Literature Review. - PubMed - NCBI. journal of orthopaedic & sports physical therapy, 45(8):576–584, 2015.
[11] Roy T H Cheung and Irene S. Davis. Landing pattern modification to improve patellofemoral pain in runners: a case series. The Journal of orthopaedic and sports physical therapy, 41 12:914–9, 2011.
[12] B Noehren, J Scholz, and I Davis. The effect of real-time gait retraining on hip kinematics, pain and function in subjects with patellofemoral pain syndrome. British Journal of Sports Medicine, 45(9):691–696, 2011.
[13] Richard W Willy, John P Scholz, and Irene S Davis. Mirror gait retraining for the treatment of patellofemoral pain in female runners. Clinical biomechanics (Bristol, Avon), 27(10):1045– 1051, December 2012.
[14] Adam C. Clansey, Michael Hanlon, Eric S. Wallace, Alan Nevill, and Mark J. Lake. Influence of tibial shock feedback training on impact loading and running economy. Medicine and science in sports and exercise, 46 5:973–81, 2014.
[15] Brian Noehren, Irene Davis, and Joseph Hamill. ASB clinical biomechanics award winner 2006 prospective study of the biomechanical factors associated with iliotibial band syn- drome. Clinical Biomechanics (Bristol, Avon), 22(9):951–956, November 2007.
[16] Brian Noehren, Joseph Hamill, and Irene Davis. Prospective evidence for a hip etiology in patellofemoral pain. Medicine and Science in Sports and Exercise, 45(6):1120–1124, June 2013.
[17] Richard B. Souza and Christopher M. Powers. Differences in hip kinematics, muscle strength, and muscle activation between subjects with and without patellofemoral pain. The Journal of Orthopaedic and Sports Physical Therapy, 39(1):12–19, January 2009.
[18] Angela R. Diebal, Robert Gregory, Curtis Alitz, and J. Parry Gerber. Effects of forefoot run- ning on chronic exertional compartment syndrome: a case series. International Journal of Sports Physical Therapy, 6(4):312–321, December 2011.
[19] Angela R. Diebal, Robert Gregory, Curtis Alitz, and J. Parry Gerber. Forefoot running im-