The second subquestion was: Does the correction of the global movement lead to a more accurate and
therefore more reproducible assessment of the videofluoroscopy? After doing our research, it turned out
that our results cannot provide a definite answer to the second subquestion. We cannot guarantee that the nullification of the global head movement will lead to a more accurate and reproducible assessment. To establish this, it would require further research in which for example the edited videos will be used to measure the amount of contrast medium in the different regions of the swallowing process, as described in the problem definition. Since the swallowing process itself is not affected and global head movement is eliminated, we are confident that the aim of measuring the amount of contrast in the trachea and oesophagus is within reach. Further research could enable us to say for sure that the correction of the global movement will lead to a more accurate and therefore more reproducible assessment of the videofluoroscopy recordings.
Chapter 9
Recommendations
I
n our discussion, we acknowledge that some improvements could be made. In this section wewant to elaborate on these improvements and make a few recommendations concerning our research. First of all, we recommend further research with videos of patients. Taking into account that our method is only tested on videos of healthy subjects, we cannot guarantee our method will be suitable as a first step of ultimately making the assessment of VFS automatic. As it is our believe that patients, who have more trouble with swallowing than healthy subjects, will show more undesired head movements. We did not compare thick and thin consistencies intrasubjectively, because this was not part of our goal to nullify the head movement of the patient. Therefore, we recommend to compare different consistencies in videos of dysphagic patients in further research.
Some videos of our subjects showed more anatomical structures, like vertebrae, and therefore contained more regions we could segment. The amount of visible vertebrae and jawbones differed intersubjectively and intrasubjectively. Therefore, we have a few recommendations for clinical use. First of all, the patient should move as little as possible while swallowing. Furthermore, to obtain the best results using non-rigid image registration, the VFS videofragment should at least contain the following anatomical structures: four vertebrae, the jaw and the orbit. Those structures are needed to produce a satisfying outcome, because our validation video showed that placing more segmented regions results in a better fixation of the head. Bear in mind, the recommended settings should not affect the assessment of the VFS negatively, because important information about the contrast medium in the trachea or oesophagus cannot be lost.
Aside from the above, we have a few recommendations for further research. First of all, we recommend the transformation of all frames instead of only five. Given our limited time, we were not able to edit each frame of every video. We recommend to use segmentation and mesh-based warping in every frame, accordingly making the results more clearly defined. Therefore, as already mentioned in the background, it would be preferable to use automatic segmentation. This would be less time-consuming and errors, due to manual segmentation, could be avoided. In order to validate the clinical relevance of the edited frames, the edited and unedited videos need to be assessed and compared by a SLP. For example, the elevation of the hyoid bone is an important movement of the swallowing function. Therefore it is essential to analyse whether the editing did not deform this important anatomical structure.
Literature research has shown that radial basis functions would probably be suited as a transformation model to nullify the global movement of the subject’s head. Therefore, we recommend to examine other radial basis functions, such as thin plate splines, for the transformation of the video fragments, because it might provide better results than the mesh-based warping algorithm [29, 32, 33, 61].
In the future the elimination of the global head movement could be used to determine which percentage of the bolus remains in the oral cavity or oropharynx. Better prediction of functional loss for patients with head and neck cancer would be possible.
Chapter 10
Acknowledgements
First of all, we would like to thank our supervisors for their time and effort in guiding us and providing us with feedback, especially F. van der Heijden and L. van der Molen. We would also like to thank the medical staff in the Antoni van Leeuwenhoek Hospital in Amsterdam, for their help, feedback and time to show us the clinical practice. Finally, we want to thank our friends and family who have supported us during our research.
References
[1] RNH Datawarehouse. populatieopbouw slikproblemen, 2013. [2] Med-Info-Groep. Dysfagie, 2011.
[3] Hanneke Kalf. Dysfagie, 2013.
[4] A. Kreeft. Functional inoperability of oral and oropharyngeal cancer, Chapter 1. Thesis, 2013. [5] P. Garcia-Peris, L. Paron, C. Velasco, C. de la Cuerda, M. Camblor, I. Breton, H. Herencia,
J. Verdaguer, C. Navarro, and P. Clave. Long-term prevalence of oropharyngeal dysphagia in head and neck cancer patients: Impact on quality of life. Clinical Nutrition, 26(6):710–717, 2007. [6] Antoni van Leeuwenhoek. Hoofd-halskanker, 2015.
[7] Association of Comprehensive Cancer Centers National Cancer Recording. Cijfers over kanker, 2011. (IKNL).
[8] Intergraal Kankercentrum Nederland. Meest voorkomende soorten kanker, 2015. [9] Amy Reinstein. The four phases of the normal adult swallow process, 2010.
[10] Raj K. Goyal and Hiroshi Mashimo. Physiology of oral, pharyngeal, and esophageal motility, 2006. [11] Koichiro Matsuo and Jeffrey B. Palmer. Anatomy and physiology of feeding and swallowing normal and abnormal. Physical medicine and rehabilitation clinics of North America, 19(4):691–707, 2008. [12] Normal swallow function, 2003.
[13] Jeri A. Logemann. Evaluation and Treatment of Swallowing Disorders. pro-ed, Austin, Texas, 1998. [14] Jeri A. Logemann. Behavioral management for oropharyngeal dysphagia. Folia Phoniatr Logop,
51(4-5):199212, 1999.
[15] Lisette van der Molen, Maya A van Rossum, Lori M Burkhead, Ludi E Smeele, and Frans J M. Hilgers. Functional outcomes and rehabilitation strategies in patients treated with chemoradiother- apy for advanced head and neck cancer: a systematic review. European Archives of Oto-Rhino-
Laryngology, 266(6):889–900, 2009.
[16] E. C. Ward and C. J. van As-Brooks. Head and Neck Cancer, Treatment, Rehabilitation, and
Outcomes. Plural Publishing, San Diego, 2007.
[17] Radiological Society of North America. Video fluoroscopic swallowing exam (vfse), 2013. (RSNA). [18] Lisette van der Molen. Introductiegesprek in het antoni van leeuwenhoek met medisch begeleider
lisette van der molen, 2015.
[19] Vinidh Paleri, Justin W. G. Roe, Primo Strojan, June Corry, Vincent Grgoire, Marc Hamoir, Avraham Eisbruch, William M. Mendenhall, Carl E. Silver, Alessandra Rinaldo, Robert P. Takes, and Alfio Ferlito. Strategies to reduce long-term postchemoradiation dysphagia in patients with head and neck cancer: An evidence-based review. Head & Neck, 36(3):431–443, 2014.
48 References
[20] Lisette van der Molen, MayaA. van Rossum, LoriM. Burkhead, LudiE. Smeele, CoenR.N. Rasch, and FransJ.M. Hilgers. A randomized preventive rehabilitation trial in advanced head and neck cancer patients treated with chemoradiotherapy: Feasibility, compliance, and short-term effects.
Dysphagia, 26(2):155–170, 2011.
[21] Gustavo Nader Marta, Valter Silva, Heloisa de Andrade Carvalho, Fernando Freire de Arruda, Samir Abdallah Hanna, Rafael Gadia, Joo Luis Fernandes da Silva, Sebastio Francisco Miranda Correa, Carlos Eduardo Cintra Vita Abreu, and Rachel Riera. Intensity-modulated radiation therapy for head and neck cancer: Systematic review and meta-analysis.Radiotherapy and Oncology, 110(1):9 – 15, 2014.
[22] D. Rueckert and P. Aljabar. Nonrigid registration of medical images: Theory, methods, and applications [applications corner]. Signal Processing Magazine, IEEE, 27(4):113–119, 2010.
[23] Mark A. Haidekker. Chapter 11: Image Registration, book section 11, pages 350–385. Hoboken, NJ : John Wiley & Sons, 2011.
[24] WR Crum, T Hartkens, and DLG Hill. Non-rigid image registration: theory and practice. The
British Journal of Radiology, 77(suppl2):S140− −S153, 2004.
[25] Vijay Luthra. interpolation, 2015.
[26] Linda G Shapiro and George C Stockman. Computer Vision: chapter 10. Pearson Education (Us), 1 edition, 2001.
[27] Neeraj Sharma and Lalit M. Aggarwal. Automated medical image segmentation techniques. J Med Phys, 35(1):3–14, 2010.
[28] Mark J. L. Orr. Radial basis function networks, 1996.
[29] M. Fornefett, K. Rohr, and H. S. Stiehl. Radial basis functions with compact support for elastic registration of medical images. Image and Vision Computing, 19(12):87–96, 2001.
[30] James Ramsay and B. W. Silverman. Functional Data Analysis. Springer Series in Statistics. Springer-Verlag, New York, 2 edition, 2005.
[31] A. Ardeshir Goshtasby.Image Registration. Advances in Computer Vision and Pattern Recognition. Springer, London, 2012.
[32] Fred L Bookstein. Principal warps: Thin-plate splines and the decomposition of deformations.IEEE
Transactions on Pattern Analysis and Machine Intelligence, 11(6):567–585, 1989.
[33] K Rohr. Spline-based elastic image registration. Proc. Appl. Math. Mech., 3:3639, 2003.
[34] Mark Holden. A review of geometric transformations for nonrigid body registration. IEEE Trans.
Med. Imaging, 27(1):111–128, 2008.
[35] Igor Yanovsky. Unbiased Nonlinear Image Registration. Thesis, 2008.
[36] William M Wells, Alan Colchester, Scott Delp, Yongmei Wang, and Lawrence H Staib.Elastic model
based non-rigid registration incorporating statistical shape information, volume 1496 ofLecture Notes
in Computer Science, pages 1162–1173. Springer Berlin Heidelberg, 1998.
[37] Themis P. Exarchos, Athanasios Papadopoulos, and Dimitrios I. Fotiadis. Handbook of Research on
Advanced Techniques in Diagnostic Imaging and Biomedical Applications. IGI Global, Hershey, PA,
References 49
[38] Roger Claypoole, Jim Lewis, Srikrishna Bhashyam, and Kevin Kelly. Elec 539, image morphing, 1997.
[39] Pramod Potluri, Krishna Sagiraju, and Venkatachalem Tubati. Image morphing: Feature based, view and mesh, 2004.
[40] Mark Steyvers. Morphing techniques for manipulating face images. Behavior Research Methods,
Instruments, & Computers, 31(2):359–369, 1999. Behavior Research Methods, Instruments, &
Computers 0743-3808.
[41] George Wolberg. Image morphing: a survey. The Visual Computer, 14(8-9):360–372, 1998. The Visual Computer 0178-2789.
[42] Sky Mckinley and Megan Levine. Cubic spline interpolation.
[43] A.M. Kreeft. Functional inoperability of oral and oropharyngeal cancer, Chapter 4. Thesis, 2013. [44] Karen H. O’Neil, Mary Purdy, Janice Falk, and Lanelle Gallo. The dysphagia outcome and severity
scale. Dysphagia, 14(3):139–145, 1999.
[45] Bonnie Martin-Harris, MartinB Brodsky, Yvonne Michel, DonaldO Castell, Melanie Schleicher, John Sandidge, Rebekah Maxwell, and Julie Blair. Mbs measurement tool for swallow impairmentmbsimp: Establishing a standard. Dysphagia, 23(4):392–405, 2008.
[46] Anne Marijn Kreeft, Lisette van der Molen, Frans Hilgers, and Alfons J Balm. Speech and swallowing after surgical treatment of advanced oral and oropharyngeal carcinoma: a systematic review of the literature. European Archives of Oto-Rhino-Laryngology, 266(11):1687–1698, 2009.
[47] L Kuil. 2D Motion Analysis of Swallowing in Videofluoroscopy. Thesis, 2013. [48] ChattanoogaGroup. Dysphagia fact sheet, 2005.
[49] Margaret Walshe. Oropharyngeal dysphagia in neurodegenerative disease.Journal of Gastroenterol-
ogy and Hepatology Research, 3(10):1265–1271, 2014.
[50] Richard D. Wilson and Evan C. Howe. A cost-effectiveness analysis of screening methods for dysphagia after stroke. PM&R, 4(4):273–282, 2012.
[51] Mission statement of the virtual therapy consortium.
[52] MatlabR . version 8.5.0.197613 (R2015a). The MathWorks Inc., Natick, Massachussetts, 2015.
[53] MatlabR . Modifying Images,version 8.5.0.197613 (R2015a). The MathWorks Inc., Natick,
Massachussetts, 2015.
[54] MatlabR . Control Point Registration,version 8.5.0.197613 (R2015a). The MathWorks Inc., Natick, Massachussetts, 2015.
[55] MatlabR . Automatic registration, version 8.5.0.197613 (R2015a). The Mathworks Inc., Natick, Massachussetts, 2015.
[56] Naive. Mesh-based image warping, 2014.
[57] MatlabR . ROI-Based Processing, version 8.5.0.197613. The MathWorks Inc., Natick,
Massachussetts, 2015.
[58] MatlabR . Image Segmentation, version 8.5.0.197613 (R2015a). The MathWorks Inc., Natick,
50 References
[59] F. Van der Heijden. Function ut contourfft, 2006.
[60] MatlabR . GPU Computing, version 8.5.0.197613. The Mathworks Inc., 2015 2015.
[61] L. Zagorchev and A. Goshtasby. A comparative study of transformation functions for nonrigid image registration. Image Processing, IEEE Transactions on, 15(3):529–538, March 2006.
Appendix A
Results
T
heresults we obtained by editing the videofluoroscopy videos are examined in the discussion and conclusion. The edited videos are classified and named as followed:• Manual selection of landmarks
– Landmarks-less-30: ∗ Subject1A-less30.avi ∗ Subject2A-less30.avi – Landmarks-more-30 ∗ Subject1A-more30.avi ∗ Subject2A-more3.avi – Landmarks-incl-Jaw ∗ Subject1A-Jaw.avi ∗ Subject2A-Jaw.avi • Segmentation – Subject1A-Segmentation.avi – Subject1B-Segmentation.avi – Subject2A-Segmentation.avi – Subject2B-Segmentation.avi – Subject3A-Segmentation.avi – Subject4A-Segmentation.avi – Validation ∗ Subject4A-Segmentation10frames.avi ∗ Subject4A-AbsSegmentation10frames.avi ∗ Subject4A-AbsOriginal.avi
These edited video files can be obtained by contacting the group members of this research. The remainder of the results are shown in the tables in this appendix.
52 Appendix A. Results
Table A.1: Transformed frames of subject 1B, frames 1 vs. 2, 1 vs. 3, 1 vs. 4, 1 vs. 5
Subject 1B 1 vs. 2 1 vs. 3 Unedited Segmentation Subject 1B 1 vs. 4 1 vs. 5 Unedited Segmentation
53
Table A.2: Transformed frames of subject 2B, frames 1 vs. 2, 1 vs. 3, 1 vs. 4, 1 vs. 5
Subject 2B 1 vs. 2 1 vs. 3 Unedited Segmentation Subject 2B 1 vs. 4 1 vs. 5 Unedited Segmentation
54 Appendix A. Results
Table A.3: Transformed frames of subject 3A, frames 1 vs. 2, 1 vs. 3, 1 vs. 4, 1 vs. 5
Subject 3A 1 vs. 2 1 vs. 3 Unedited Segmentation Subject 3A 1 vs. 4 1 vs. 5 Unedited Segmentation
Appendix B
Mesh-based warping script
The first script defines the function ‘mesh-based warping’ [56].
1 f u n c t i o n [Q] = m e s h b a s e d w a r p i n g ( s o u r c e I m a g e , t a r g e t I m a g e , p o s s s o u r c e , p o s s t a r g e t , a l p h a , b e t a )
2 % warp t h e s o u r c e image t o a t a r g e t u s i n g a mesh−b a s e d w a r p i n g method . 3 % s o u r c e I m a g e : RGB f o r m a t i n p u t s o u r c e image
4 % t a r g e t I m a g e : RGB f o r m a t i n p u t t a r g e t image which h a s t h e same s i z e w i t h s o u r c e image 5 % n u m p o i n t s : number o f f e a t u r e p o i n t s 6 % a l p h a : c o e f f i c i e n t 7 % a l p h a = 1 −−> o u t p u t = t a r g e t image 8 % a l p h a = 0 −−> o u t p u t = s o u r c e image 9 10 xxA=p o s s s o u r c e ( 1 , : ) ; 11 yyA=p o s s s o u r c e ( 2 , : ) ; 12 xA=xxA ( : ) ; 13 yA=yyA ( : ) ; 14 15 xxB=p o s s t a r g e t ( 1 , : ) ; 16 yyB=p o s s t a r g e t ( 2 , : ) ; 17 xB=xxB ( : ) ; 18 yB=yyB ( : ) ; 19 20 A=s o u r c e I m a g e ; 21 B=t a r g e t I m a g e ; 22 23 h e i g h t = s i z e( s o u r c e I m a g e , 1 ) ; 24 wi d th = s i z e( s o u r c e I m a g e , 2 ) ; 25 26 h e i g h t 2 = s i z e( t a r g e t I m a g e , 1 ) ; 27 w i d t h 2 = s i z e( t a r g e t I m a g e , 2 ) ; 28 29 i f( h e i g h t ˜= h e i g h t 2 | | w id th˜=w i d t h 2 ) 30 e r r o r(’ I n p u t i m a g e s s h o u l d have same s i z e . ’) 31 end 32 33 xA = [ xA ; 1 ; w id th ; wi d th ; 1 ] ; 34 yA = [ yA ; 1 ; 1 ; h e i g h t ; h e i g h t ] ; 35 36 xB = [ xB ; 1 ; w id th ; wi dt h ; 1 ] ; 37 yB = [ yB ; 1 ; 1 ; h e i g h t ; h e i g h t ] ; 38 39 % c r e a t e a i n t e r m e d i a t e g r i d 40 C = z e r o s( h e i g h t , width , 3 ) ; 41 xC = a l p h a∗xA + (1−a l p h a )∗xB ; 42 yC = a l p h a∗yA + (1−a l p h a )∗yB ; 43 t r i C = d e l a u n a y ( xC , yC ) ; 44 n t r i = s i z e( t r i C , 1 ) ; 45 46 % a l l o c a t e memory f o r x , y c o o r d i n a t o r s 47 xCA = z e r o s( h e i g h t , w id th ) ; 48 yCA = z e r o s( h e i g h t , w id th ) ; 49 xCB = z e r o s( h e i g h t , wi d th ) ; 50 yCB = z e r o s( h e i g h t , wi d th ) ;
56 Appendix B. Mesh-based warping script 51 [ X, Y ] = m e s h g r i d( 1 : width , 1 : h e i g h t ) ; 52 53 % warp t h e i n t e r m e d i a t e g r i d t o s o u r c e and t a r g e t g r i d 54 f o r k = 1 : n t r i 55 [ w1 , w2 , w3 , r ] = i n T r i (X, Y, xC ( t r i C ( k , 1 ) ) , yC ( t r i C ( k , 1 ) ) , xC ( t r i C ( k , 2 ) ) , yC ( t r i C ( k , 2 ) ) , xC ( t r i C ( k , 3 ) ) , yC ( t r i C ( k , 3 ) ) ) ; 56 w1 ( ˜ r ) =0; 57 w2 ( ˜ r ) =0; 58 w3 ( ˜ r ) =0;
59 xCA = xCA + w1 .∗xA( t r i C ( k , 1 ) ) + w2 .∗xA( t r i C ( k , 2 ) ) + w3 .∗xA( t r i C ( k , 3 ) ) ; 60 yCA = yCA + w1 .∗yA( t r i C ( k , 1 ) ) + w2 .∗yA( t r i C ( k , 2 ) ) + w3 .∗yA( t r i C ( k , 3 ) ) ; 61 xCB = xCB + w1 .∗xB ( t r i C ( k , 1 ) ) + w2 .∗xB ( t r i C ( k , 2 ) ) + w3 .∗xB ( t r i C ( k , 3 ) ) ; 62 yCB = yCB + w1 .∗yB ( t r i C ( k , 1 ) ) + w2 .∗yB ( t r i C ( k , 2 ) ) + w3 .∗yB ( t r i C ( k , 3 ) ) ; 63 end
64
65 % i n t e r p o l a t e e a c h p o i n t by u s i n g ’ i n t e r p 2 ’ f u n c t i o n 66 VCA( : , : , 1 ) = i n t e r p 2(X, Y, d o u b l e (A ( : , : , 1 ) ) ,xCA, yCA) ; 67 VCA( : , : , 2 ) = i n t e r p 2(X, Y, d o u b l e (A ( : , : , 2 ) ) ,xCA, yCA) ; 68 VCA( : , : , 3 ) = i n t e r p 2(X, Y, d o u b l e (A ( : , : , 3 ) ) ,xCA, yCA) ; 69 70 VCB ( : , : , 1 ) = i n t e r p 2(X, Y, d o u b l e (B ( : , : , 1 ) ) ,xCB , yCB) ; 71 VCB ( : , : , 2 ) = i n t e r p 2(X, Y, d o u b l e (B ( : , : , 2 ) ) ,xCB , yCB) ; 72 VCB ( : , : , 3 ) = i n t e r p 2(X, Y, d o u b l e (B ( : , : , 3 ) ) ,xCB , yCB) ; 73 74 % c r o s s−d i s s o l v e 75 C = b e t a∗VCA + (1−b e t a )∗VCB; 76 77 % c o n v e r t d o u b l e t o u i n t 8 f o r m a t and d i s p l a y 78 %imshow ( u i n t 8 (C) ) ; 79 Q=u i n t 8 (C) ; 80 end 81
82 f u n c t i o n [ w1 , w2 , w3 , r ] = i n T r i ( vx , vy , v0x , v0y , v1x , v1y , v2x , v2y )
83 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 84 % i n T r i c h e c k s w h e t h e r i n p u t p o i n t s ( vx , vy ) a r e i n a t r i a n g l e whose
85 % v e r t i c e s a r e ( v0x , v0y ) , ( v1x , v1y ) and ( v2x , v2y ) and r e t u r n s t h e l i n e a r 86 % c o m b i n a t i o n w e i g h t , i . e . , vx = w1∗v0x + w2∗v1x + w3∗v2x and
87 % vy = w1∗v0y + w2∗v1y + w3∗v2y . I f a p o i n t i s i n t h e t r i a n g l e , t h e 88 % c o r r e s p o n d i n g r w i l l be 1 and o t h e r w i s e 0 . 89 % 90 % T h i s f u n c t i o n a c c e p t s m u l t i p l e p o i n t i n p u t s , e . g . , f o r two p o i n t s ( 1 , 2 ) , 91 % ( 2 0 , 3 0 ) , vx = ( 1 , 2 0 ) and vy = ( 2 , 3 0 ) . I n t h i s c a s e , w1 , w2 , w3 and r w i l l 92 % be v e c t o r s . The f u n c t i o n o n l y a c c e p t s t h e v e r t i c e s o f one t r i a n g l e . 93 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 94 v0x = repmat ( v0x , s i z e( vx , 1 ) , s i z e( vx , 2 ) ) ;
95 v0y = repmat ( v0y , s i z e( vx , 1 ) , s i z e( vx , 2 ) ) ; 96 v1x = repmat ( v1x , s i z e( vx , 1 ) , s i z e( vx , 2 ) ) ; 97 v1y = repmat ( v1y , s i z e( vx , 1 ) , s i z e( vx , 2 ) ) ; 98 v2x = repmat ( v2x , s i z e( vx , 1 ) , s i z e( vx , 2 ) ) ; 99 v2y = repmat ( v2y , s i z e( vx , 1 ) , s i z e( vx , 2 ) ) ;
100 w1 = ( ( vx−v2x ) .∗( v1y−v2y ) − ( vy−v2y ) .∗( v1x−v2x ) ) . / . . . 101 ( ( v0x−v2x ) .∗( v1y−v2y ) − ( v0y−v2y ) .∗( v1x−v2x )+e p s) ; 102 w2 = ( ( vx−v2x ) .∗( v0y−v2y ) − ( vy−v2y ) .∗( v0x−v2x ) ) . / . . . 103 ( ( v1x−v2x ) .∗( v0y−v2y ) − ( v1y−v2y ) .∗( v0x−v2x )+e p s) ; 104 w3 = 1 − w1 − w2 ;
105 r = ( w1>=0) & ( w2>=0) & ( w3>=0) & ( w1<=1) & ( w2<=1) & ( w3<=1) ; 106 end
57
The second script uses the previous stated function, mesh-based warping, and is used to obtain the warped frames. 1 c l e a r a l l 2 c l o s e a l l