MUDr.Andrej Thurzo,Ph.D.
Oddelenie ortodoncie, Klinika stomatoloÂgie a maxilofaciaÂlnej chirugie, LekaÂrska fakulta KomenskeÂho univerzity a Institut onkologii Sv.Elizabety , Bratislava, Slovensko
Department of Orthodontics, Clinic of Stomatology and Craniofacial Surgery, Medical Faculty of Comenius Uni- versity, Oncological Institute of St.Elizabeth, Bratislava, Slovakia
Abstrakt
CÏlaÂnok prinaÂsÏa pohl'ad do oblasti skenovania tvaÂre v ortodoncii ako aj fuÂzie takeÂhoto 3D zaÂznamu s celohlavo-
vyÂm CBCT iCATTM skenom. Autor popisuje vlastne skuÂsenosti s 3D tlacÏou takeÂhoto kombinovaneÂho modelu
a orientacÏne posudzuje presnost' reprodukcie medzi reaÂlnym modelom, virtuaÂlnym 3D modelom a vytlacÏenyÂm modelom. Vysvetl'uje tiezÏ princõÂpy a rozdiely medzi aktõÂvnou a pasõÂvnou stereofotogrametriou a tiezÏ porovnaÂva
skuÂsenosti medzi jednotlivyÂmi stereoskopickyÂmi kamerami: 3DMDTM, 3D-ShapeTMa DI3DTM. CÏlaÂnok tiezÏ venuje
pozornost' perspektõÂvam klinickeÂho vyuzÏitia uvedenyÂch technologickyÂch postupov(Ortodoncie 2011, 20, cÏ. 4, s. 228-
236).
Abstract
The article reviews the face-scanning procedures and technologies in orthodontics, as well as the fusion of
such face-scans with CBCT iCATTM scans. The author gives his own experience with 3D print of the fused
face-CBCT model, and evaluates in brief the accuracy of a reproduction in vivo, a virtual 3D model, and a printed 3D model. The basic principles and differences between active and passive stereophotogrammetry are explai-
ned, and the experience with different stereoscopic cameras 3DMDTM, 3D-ShapeTM, and DI3DTMis discussed.
The article also gives the current view on perspectives and the use of the technologies described in clinical ortho-
dontics(Ortodoncie 2011, 20, No. 4, p. 228-236).
Kl'uÂcÏove slovaÂ: 3D tlacÏ v ortodoncii, CBCT, sken tvaÂre, stereofotogrametria, AM, aditõÂvna vyÂroba
Key words: 3D print in orthodontics, CBCT, face scan, stereophotogrammetry, AM, additive manufacturing UÂvod
Ciel'om tohto cÏlaÂnku je oboznaÂmi t'cÏitatel'a so suÂcÏas- nyÂmi technikami skenovania tvaÂre v ortodoncii a ich vyuzÏitia v kombinaÂcii s 3D-tlacÏou. Podobne su pred- stavene prve skuÂsenostiz postupov pocÏõÂtacÏovej fuÂzi e 3D skenu tvaÂre a 3D skenu lebky (CBCT) a ich naÂsled- nej 3D tlacÏe kompozitnyÂm materiaÂlom na baÂze sadry a plastu impregnovaneÂho cyanakrylaÂtom. Porovnanie jednotlivyÂch skenovacõÂch systeÂmov, ako aj prve skuÂ- senostiso ¹zhmotnenõÂmª kombinovanyÂch virtuaÂlnych modelov tvaÂre a lebky, nemaju vyÂznam len pre plaÂno-
Introduction
The aim of the article is to inform about the current methods of face scanning in orthodontics, and the use of the methods when combined with 3D print. We bring in detail early experience with a computer as- sisted fusion of 3D face scan with 3D skull scan (CBCT) and the following 3D print using composite material produced on the basis of plaster cast and plastic im- pregnated with cyanoacrylate. The comparison of indi- vidual scanning systems, and early experience with ¹materializationª of the combined virtual models of
face and skull are used in the planning of orthognathic surgery. However, the area of clinical application will be significantly larger. Early experience is used to in- troduce the new technological procedures as well as to bring the new technological procedures to the atten- tion of researchers. The main objective of the article is, therefore, to inspire more research in the area and to study other possible uses of the technologies in ortho- dontics. To fully comprehend the clinical experiment, it is necessary to understand theoretical background of 3D face scanning.
Face scanning
Stereoscopic face scanners have been available on the market for several years now. They have been con- tinuously developing, and the main reason for this is biometry. One of the most economical scan methods is laser scanning. However, laser scanning is not the best choice in case of reflexive surfaces, and it may be biologically harmful. Laser scanning is not appro- priate for scanning of sharp-edged surfaces, surfaces with transitions and gaps. In the clinical practice the accuracy of face scans is the key factor. Stereoscopic face scanning in orthodontics is used especially in re- search and in the planning of orthognathic surgery [1]. A sequence of such scans may be used e.g. in asses- sing the growth or the treatment impact on soft tissues. The scan may also help in the planning of esthetic sur- gery [2] - so far the ¹idealª face proportions were eva- luated without paying attention to the third dimension. E.g. 3DMD scanner (3dMD Ltd., USA) utilizes 4 came- ras together with the projection of a texture to evaluate a curvature (active stereophotogrammetry), in 2 milli- second exposition (Fig. 1). The whole process takes up to 10 seconds. Apart from the 3D face model, it is possible to make also a colored texture, usually from ear to ear. It is not required that the subject scanned is still; however, it is required that the head is in the po- sition desired. The scanner focal length is about 30 cm; the 3DMD accuracy is approx. 0.1 mm.
In comparison with CBCT, stereoscopic scanners offer non-invasive examination. Therefore, they are often used in assessing of a patient`s growth or in eva- luation of a therapy effect on soft tissues. However, due to the fact that they are economically rather de- manding, they are used especially in research, and are still rather rare in clinical practice.
Stereoscopic scanner DI3DTM FCS-100 (Dimen- sional Imaging Ltd., Glasgow, Scotland) is based on the passive stereophotogrammetry technique. This method makes it possible to create extremely accurate 3D models out of two pairs of photographs. The grid on face projection is not required, nor is used laser scan- ning. The result is the extraordinary fast process of vanie ortognaÂnych operaÂciõÂ, ale ich klinicka aplikaÂci a
bude podstatne sÏi rsÏia. Prve prakticke skuÂsenostimaju nielen predstavit', ale aj priblõÂzÏit'tieto technologicke po- stupy potencionaÂlnym vyÂskumnõÂkom. Ciel'om cÏlaÂnku je insÏpirovat'k d'alsÏõÂm vyÂskumom v tejto oblastia skuÂma- niu d'alsÏieho potenciaÂlu tyÂchto technoloÂgiõ vo vzt'ahu k ortodoncii. Pre pochopenie klinickeÂho experimentu je potrebne porozumeniu teoretickeÂho pozadia 3D skenovania tvaÂre.
Skenovanie tvaÂre
Stereoskopicke skenery tvaÂre su komercÏne k dispo- zõÂcii uzÏ niekol'ko rokov. Jeden z hlavnyÂch doÃvodov ich neustaÂleho vyÂvoja je biometria. Jeden z najekonomic- kejsÏõÂch spoÃsobov skenovania je skenovanie laserom, ktore ma vsÏak slabiny pri snõÂmanõ zrkadliacich sa po- vrchov alebo je nebezpecÏne z biologickeÂho hl'adiska. Laserove skenovanie je tiezÏ problematicke prisnõÂmanõ povrchov s ostryÂmihranami, prechodmicÏimedzerami. Presnost' je pre vyuzÏitel'nost' skenov tvaÂre v klinickej praxi, kl'uÂcÏovaÂ. Stereoskopicky skener tvaÂre sa v suÂ- cÏasnostiv ortodonciivyuzÏõÂva predovsÏetkyÂm vo vyÂ- skume a priplaÂnovanõ ortognaÂtnych chirurgickyÂch zaÂ- krokov [1]. Sekvencia takyÂchto skenov moÃzÏe byt' pou- zÏita naprõÂklad priposudzovanõ rastu alebo vplyvu liecÏby na maÈkke tkanivaÂ. Samotny sken mozÏno vyuzÏit' priplaÂnovanõÂliecÏby aj vzhl'adom na ¹ideaÂlneª proporcie estetiky tvaÂre [2]), ktore bolidoteraz posudzovane bez ohl'adu na tretõ rozmer. NaprõÂklad 3DMD skener (3dMD Ltd, USA) vyuzÏõÂva 4 kamery spolu s premietnutõÂm tex- tuÂry pre vyhodnotenie zakrivenõ (aktõÂvna stereofoto- grametria) a to vsÏetko pri 2 milisekundovej expozõÂcii (Obr. 1). Cely proces spracovania trva asi10 sekuÂnd. Okrem samotneÂho 3D modelu tvaÂre dokaÂzÏe vytvorit' aj farebnu textuÂru obvykle od ucha k uchu. SnõÂmany subjekt nemusõ ostat' nehybnyÂ, ale je potrebneÂ, aby umiestnil hlavu do zÏiadanej pozõÂcie. Skener ma ohni- skovu vzdialenost' priblizÏne 30 centimetrov. Presnost' 3DMD je priblizÏne 0.1 mm.
Stereoskopicke skenery su vyÂhodne svojou neinva- zivitou v porovnanõ s CBCT a preto su frekventovane pouzÏõÂvane prihodnotenõ rastu pacienta alebo efektu liecÏby na maÈkke tkanivaÂ. Pre svoju ekonomicku naÂrocÏ- nost' ostaÂvaju predovsÏetkyÂm v rovine vyÂskumu a do bezÏnej klinickej praxe zatial'neprenikajuÂ.
Stereoskopicky skener DI3D FCS-100 (Dimensio- nal Imaging Ltd, Glasgow, SÏkoÂtsko) vyuzÏõÂva metoÂdu pasõÂvnej stereofotogrametrie. TaÂto umozÏnÏuje vytvore- nie extreÂmne presnyÂch 3D modelov z dvoch paÂrov fo- tografiõÂ. NevyzÏaduje projekciu zÏiadnej mriezÏky na tvaÂr anizÏiadnu formu laseroveÂho skenovania. VyÂsledkom je mimoriadne ryÂchly proces skenovania prirovnatel'ny k bezÏneÂmu fotografovaniu avsÏak za pouzÏitia sÏtyroch
scanning which can be compared with common taking shots, using four calibrated digital cameras (Fig.2).
Stereophotogrammetry makes it possible to calcu- late 3D coordinates for each point of the object [3,4]. These points are set by the calculation between two photographs taken from different angles. In shots, pairs of identical points are identified. The straight line coming from the objective to a given point of a 3D object crosses the analogical straight line coming from the ot- her objective. By triangulation we are able to calculate spatial coordinates for each pixel of the given photo- graph. Triangulation is used to calculate an unknown position of the point by using known angles of straight lines (angle of the objective view) coming from the known points (the objective position), instead of the di- rect calculation of distances.
digitaÂlnych fotoaparaÂtov vo vzaÂjomnej kalibraÂci i (Obr. 2).
Stereofotogrametria umozÏnÏuje vypocÏõÂtat'3D koordi- naÂty kazÏdeÂho bodu na objekte [3, 4]. Tieto su stano- vene vyÂpocÏtom medzidvoma fotografiamiurobenyÂmi z rozdielnych pohl'adov. Na fotografiaÂch sa identifikuju paÂry identickyÂch bodov. Priamka pohl'adu z objektõÂvu na dany bod 3D objektu ma v tom bode priesecÏnõÂk s analogickou priamkou z druheÂho objektõÂvu. Pomo- cou triangulaÂci e su vypocÏõÂtane priestorove koordinaÂty pre kazÏdy pixel danej fotografie. TriangulaÂcia vypocÏõÂ- tava neznaÂmu polohu bodu pomocou znaÂmych uhlov priamok(uhol pohl'adu objektõÂvu) vychaÂdzajuÂcich zo znaÂmych bodov (poloha objektõÂvu) namiesto priameho vyÂpocÏtu vzdialenostõÂ.
V prõÂpade DI3D skenera je tak pripouzÏitõ 10 megapi- xelovyÂch fotoaparaÂtov mozÏne zmapovat' 10 milioÂnov bodov z jedneÂho stereopaÂru fotografiõÂ. Pre d'alsÏie ma- nipulaÂcie s 3D skenom tvaÂre je potrebne konvertovat' ho do polygonaÂlnej mriezÏky, cÏo vaÈcÏsÏina softveÂrov
Obr. 1. Hore: prõÂklad 3DMD skenu tvaÂre (model s textuÂrou a bez); Dole: skenovanie 3DMD stereoskopickou kamerou.
Fig. 1. Above: 3DMD face scan (with and without texture); Bellow:
Stereoscopic scanning with 3DMD. Courtesy www.3dmd.com Obr. 2. Hore: vyÂsledok DI3D skenu tvaÂre (model s textuÂrou a bez);Dole: DI3D FCS-100 systeÂm so sÏtyrmi10-megapixelovyÂmifotoapa- raÂtmivytvorõ 10 megapixelove povrchove textuÂry.
Fig. 2. Above: DI3D face scan (with and without texture); Bellow: DI3D FCS-100 system which is using four 10 megapixel cameras creating 20 megapixel surface texture maps. Courtesy www.DI3D.com
In case of DI3D scanner, when using 10 megapixel cameras, it is possible to map 10 millions of points in one stereo-pair of photographs. To further manipulate the 3D face scan it is necessary to convert it into a po- lygonal grid (which is possible with majority of softwa- res). The high definition texture (a face photograph) may be then mapped onto the polygon grid.
FaceSCAN3D (3D-Shape GmbH, Germany) is a ste- reoscopic scanner using active stereophotogramme- try with projection of a pattern on the face scanned (Fig.3). The scanner accuracy is approx. 0.1 mm, and the scan of the whole face, from ear to ear (180) takes 0.8 second (Fig.3).
3D analysis and 4D record of a face
Face scans analyses can be based on the mutual differences between the individual scans, the differen- ces may be visualized, measured in terms of volume, measured in terms of surface and linear distances of points, and also face symmetry may be evaluated in 3D. 4D record is a time sequence of 3D face scans. It is usually based on the principle of passive steropho- togrammetry, i.e. it utilizes a pair scanning by two ca- meras affected in the shades of grey, sometimes in ad- dition with a synchronized camera scanning surface texture. The result is a time sequence of polygon grids. The sequence can be used in e.g. the analysis of ma- stication movements.
Advantages of passive stereophotogrammetry in comparison with 3D scanning
Scanning is usually associated with a laser scan- ning. 3D scanning (especially 3D laser scanning) is considered to be an extremely precise method of a 3D record of a surface. In scanning a human face the passive stereophotogrammetry has several ad- vantages in comparison with 3D scanning:
- Immediate capture of an image: it is possible to capture living mobile organisms
- High definition: the basic form is represented by high-definition photographs.
Advantages of passive stereophotogrammetry in comparison with active stereophotogrammetry
Active and passive stereophotogrammetry are ra- ther similar. However, the active one relies on ¹structu- red lightª i.e. it works with a projection of a known pa- ttern onto a face. By means of backward scanning of the pattern, the incurvation is measured. Active ste- reophotogrammetry shows often problems in scan- ning darker complexion, bearded faces, etc. In such areas the deformation of the projecting image is not vi- sible, and thus we lack information on spatial incurva- ture.
umozÏnÏuje. Na tuÂto polygonaÂlnu mriezÏku moÃzÏe byt' na- mapovana textuÂra s vysokyÂm rozlõÂsÏenõÂm (fotografia tvaÂre).
FaceSCAN3D(3D-Shape GmbH, Nemecko) je ste-
reoskopicky skener vyuzÏõÂvajuÂciaktõÂvnu stereofotogra- metriu s premietnutõÂm obrazca na skenovanu tvaÂr (Obr. 3). Presnost' tohto skenera je priblizÏne 0,1 mm a doba skenovania celej tvaÂre, od ucha k uchu (>180, je 0,8 sekundy (Obr. 3).
3D analyÂza a 4D zaÂznam tvaÂre
TvaÂrove skeny mozÏno analyzovat' na zaÂklade vzaÂjomnyÂch rozdielov medzi jednotlivyÂmi skenmi, vi- zualizovat' tieto rozdiely, objemovo merat' vzaÂjomne rozdiely , merat' plochy a lineaÂrne vzdialenosti bodov tiezÏ trojrozmerne hodnotit' symetriu tvaÂre. 4D zaÂznam tvaÂre predstavuje cÏasovu sekvenciu 3D zaÂznamov tvaÂre. Obvykle funguje na princõÂpoch pasõÂvnej stereo- fotogrametrie, teda vyuzÏõÂva paÂrove snõÂmanie dvoch kamier v odtienÏoch sÏedej a prõÂpadne doplnkovu synch- ronizovanu kameru pre zachytenie povrchovej textuÂry.
Obr. 3. Hore: sken tvaÂre systeÂmom 3D-Shape (model s textuÂrou a bez). V strede: premietana textuÂra priskenovanõÂ. Dole: FaceS- CAN3D od 3D-Shape.
Fig. 3. Above: face scan (with and without texture). MIDDLE: texture overlay during scanning. Bellow: FaceSCAN3D from 3D-Shape. Courtesy www.3d-shape.com
Our experience
Material used and technologies applied
We performed the above described procedure - be- ginning with scanning and ending with 3D print. Two models were scanned and printed (face scan and skull scan).
For 3D RTG scanning we used CBCT type iCAT (Imaging Sciences, USA); the scan extent correspon- ded to the maximum size of the field of view (FOV) (Ta- ble 1) of the device, and consisted of 576 scans with the definition of 0.3 mm. CNCT scan contained arte- facts due to amalgam fillings, the shift between the first and the last scan of CBCT did not exceed 2 mm.
3D record from CBCT was reconstructed with the software MD Studio/Invivo 5 (Anatomage, USA). Inten- tionally, we did not perform any adjustment of the vo- lume, no trimming or deformation of the final STL grid. The objective was to evaluate the immediate material reproduction of CBCT scan without any alterations.
For printing we used the 3D printer Spectrum Z510 (Z Corporation, USA), that is able to produce high-de- finition prototypes in full colour relatively fast. The prin- ted models of a face scan and a skull scan were redu- ced to 70% and were printed in black and white.
The material costs did not exceed 50 EUR/1 skull (6 pieces at the same time) / according to the official data given by the company dealing with 3D print (Hmat s.r.o., Slovakia). In this size it was possible to print as many as 6 skulls at the same time, and the printing took 10 hours. In case of printing 100% size of a CBCT re- cord, it was possible to print 4 skulls at the same time and the printing process took 18 hours. In printing of 4 skulls at the same time, the material costs per one skull did not exceed 100 EUR.
3D printer Spectrum Z510 has the following para- meters:
Time of line-up: 2-4 layers per minute Maximum line-up size: 254 x 356 x 203
Layer thickness: adjustable by the user at time of printing / from 0.089 to 0.203 mm
VyÂsledkom je cÏasova sekvencia polygoÂnovyÂch mrie- zÏok. TakuÂto sekvenciu mozÏno vyuzÏit'naprõÂklad priana- lyÂze zÏuvacõÂch pohybov.
VyÂhody pasõÂvnej stereofotogrametrie pred 3D ske- novanõÂm
Technika skenovania sa obvykle spaÂja s laserovyÂm skenovanõÂm. 3D skenovanie (predovsÏetkyÂm 3D lase- roveÂ) sa povazÏuje za extreÂmne presny spoÃsob 3D za- znamenania povrchu. Pri skenovanõ l'udskej tvaÂre ma pasõÂvna stereofotogrametria niekol'ko vyÂhod pred 3D skenovanõÂm
- OkamzÏite zachytenie obrazu: mozÏne zachytit'zÏiveÂ, pohybujuÂce sa organizmy.
- Vysoke rozlõÂsÏenie: ZaÂkladom su fotografie vo vy- sokom rozlõÂsÏenõÂ.
VyÂhody pasõÂvnej stereofotogrametrie pred aktõÂvnou AktõÂvna a pasõÂvna stereofotogrametria su sipo- dobneÂ. AktõÂvna sa vsÏak spolieha na ¹Structured Lightª teda vyuzÏõÂva premietanie znaÂmeho obrazcu na tvaÂr. Potom spaÈtnyÂm nasnõÂmanõÂm obrazcu zistõ zakrivenie. AktõÂvna stereofotogrametria ma cÏasto probleÂmy pri skenovanõ tvaÂrõ tmavsÏej pleti, zarastenyÂch tvaÂrõÂ, fuÂzov a podobne. V tyÂchto oblastiach nie je deformaÂcia pre- mietaneÂho obrazca viditel'na a chyÂba tak informaÂci a o priestorovom zakrivenõÂ.
NasÏe skuÂsenosti
PouzÏity materiaÂl a technoloÂgi e
Bol uskutocÏneny vysÏsÏie popõÂsany proces - od zaÂ- znamu po 3D tlacÏ. Zoskenovane a vytlacÏene bolidva modely (sken tvaÂre a sken skeletu lebky).
Pre 3D RTG skenovanie bol vyuzÏity CBCT typ iCAT (Imaging Sciences, USA)., rozsah skenu zodpovedal maximaÂlnej vel'kostiskenovaneÂho pol'a (Field of View - FOV) (Tab. 1) tohto prõÂstroja a bol zlozÏeny z 576 rezov prirozlõÂsÏenõ 0,3 mm. CBCT sken obsahoval artefakty spoÃsobene amalgaÂmovyÂmivyÂplnÏamia posun medzi prvou a poslednou snõÂmkou CBCT skenu nepresiahol 2mm.
Tab. 1. PouzÏite skratky Tab. 1. List of abbreviations
2D, 3D, 4D dvoj-, troj-, sÏtvor-, rozmerny two-, three-, four- dimensional AM AditõÂvna vyÂroba (Additive Manufacturing) je proces vyÂ-
roby cÏastõÂ nanaÂsÏanõÂm jednotlivyÂch vrstiev materiaÂlu na- miesto jeho orezaÂvania/odstranÏovania.
additive manufacturing is the procedure when individual layers of material are added, instead of its cutting/remo- val
CBCT Cone Beam CT, novsÏia technoloÂgia pocÏõÂtacÏovej tomo-
grafie s priblizÏne 10-naÂsobne nizÏsÏou zaÂt'azÏou Cone Beam CT, a newer technology of computer tomo-graphy, with much less load of radiation FOV vel'kost' skenovaneÂho pol'a (napr. priCBCT) - Field of
View field of view, a size of scanned field (e.g. in CBCT) STL formaÂt pocÏõÂtacÏovyÂch suÂborov typickyÂch pre stereolito-
Material: high-performance composites, direct ca- sting
Definition: 600 x 540 dpi
Dimensions and weight of the apparatus: 107 x 79 x 127 cm, 204 kg.
Definition used in the print of the above mentioned models: X,Y: 600x540dpi, Z: 0.1 mm layer (256dpi).
We used a composite material z131 based on pla- ster and plastic, impregnated with cyanoacrylate.
Out of the three scans at our disposal, we chose the scan by DI3DTM because of the homogenous and de- tailed surface grid [1]. It had a thickness of 4 mm, in which there were relief cuts complementary with corre- sponding processes of the skull. The position of a face scan and CBCT was oriented by means of fading over the CBCT surface and a corresponding face surface and the face scan itself. This orientation was set in the software mentioned above (Anatomage). Thus we could join the printed face scan and the skull scan in one whole (Fig.4).
To assess the accuracy of the reproduction, we measured and compared linear distances in each mo- del (real, virtual, and printed one):
- The longest outer and the shortest inner distance between teeth 16 and 26
- The distance between the lower incisal centre and mesial aproximal surface 36
- The distance between the lower incisal centre and mesial aproximal surface 46
Results
When assessing the accuracy of the reproduction between the real model, the virtual 3D model, and the printed model, the linear dimensions were compared - the longest outer and the shortest inner distance bet-