Introduction

Anatomical variation in intrahepatic vasculature and biliary structures challenges the surgeon.Unnecessary liver resection impacts subsequent liver function,postoperative complications and patient recovery[1,2].Augmented reality(AR)is based on the combinations of high-quality preoperative imaging data,a digital surgical platform that integrates three-dimensional(3D)image reconstruction and geometric analysis.AR generates intraoperative visual aids to assist in surgical navigation and improves the precision of liver resection[3].AR for preoperative evaluation of the relevant anatomy and physiological functions of surgical targets also predicts the outcomes of liver surgery[3,4].

By rendering transparent virtual images of organs and lesions,AR can partially compensate for a surgeon’s lack of tactile sensation,which is often experienced during video-assisted procedures,such as laparoscopic surgery.Computer-assisted surgical navigational systems that are viewed external to the surgical field of view have been reported in the literature[5].In AR-assisted liver surgery,images are superimposed,either directly on the surgical field or on images of the surgical field of view,which help the surgeon to visualize internal liver structures.Researchers have investigated the AR technology in surgical procedures and teaching and training of liver surgery[6–8].In this review,we describe AR principles,applications,advantages,and shortcomings in liver resection,and discuss the prospects of AR for improving the precision of liver surgery.

Precision liver surgery

Precision liver surgery incorporates information technology,biomedical Engineering,and digital imaging.In recent years,rapid development of both medical imaging and computer-assisted surgery has allowed surgeons to achieve greater accuracy of liver surgery[9].We have achieved our core strategy by combining maximum lesion removal,maximum organ preservation,and minimal surgical trauma to the patient[3].

将理论学习和技能训练分解成若干个子任务、子项目，以其中的某一个项目为对象，先由老师对该项目进行适当的示范，然后让学生分组围绕各自的项目进行讨论、练习并最终协作完成项目任务。例如，在“个人仪表礼仪”实训练习时，将“仪表礼仪”分成“色彩诊断”、“肤色比对”和“服装搭配”三个子项目，由老师讲解实训目的和要求，并先进行示范操作；然后将学生分成若干小组，每组学生交叉练习肤色和色彩诊断，尝试进行不同款式的服装搭配；最后，每组同学随机抽签进行三种不同场合的服饰搭配，并提交照片和实训报告。通过项目分解和协作实训使学生能掌握不同的肤色和色彩特征，并根据不同场合进行正确的服装搭配。

Compared with those of conventional technologies,AR improves visualization for surgical navigation and is an important element of precision liver surgery.In conventional video-based surgical approaches,such as laparoscopic surgery,depth perception tends to be lost when viewing two-dimensional(2D)video images generated by endoscopic cameras[10].The reduced tactile sensation and suboptimal video resolution associated with conventional video-based navigation are also problematic[11].Stereoscopic video improves spatial awareness within a laparoscopic surgical field,but the inability to rotate the stereoscopic lenses in the tangential plane confounds adjustment to the field of view.The ability to visualize mentally intrahepatic structures is arguably the most critical determinant to the success of hepatobiliary surgery.In an effort to compensate for the lack of precise information regarding the tumor structure,surgeons may overestimate the extent of resection required to ensure disease-free margins,which can damage unseen intrahepatic blood vessels and biliary ducts.Therefore,improvements in visualization technologies for both open and laparoscopic liver surgery should lead to improved clinical outcomes[3,12].

原料：低筋面粉160克、融化的黄油和细砂糖各60克、鸡蛋 150克、牛奶100克、玉米淀粉40克、泡打粉3-4克

When displayed,the partially translucent image generated by volume rendering allows the viewer to distinguish between anatomical structures with very different densities[19].Thus,volume rendering is more useful for differentiating dense tissues,such as bone,from softer tissues and fluids.However,volume rendering can allow the viewer to differentiate between different types of soft tissue if the tissues are adequately contrasted and the transfer function is well de fined[20].Volume rendering is especially useful for the reconstruction of images of bones and vascular formations,which rarely require segmentation.Examples of post-processing software used for volume rendering include OsiriX(Pixmeo,Bernex,Switzerland)and VR-Render(Research Institute Against Digestive Cancer[IRCAD],Cedex,France).

Concept and elements of AR

The AR concept involves superimposition of virtual computergenerated images onto the actual surgical field to augment the surgeon’s efforts to visualize structures that are difficult to access or are located within solid organs,such as blood vessels,biliary ducts and tumors in the liver[2,11,15].Developing an AR navigation system requires the reconstruction of 3D images from 2D CT,angiography,or magnetic resonance imaging(MRI)data.In most AR methods,the 3D virtual images are overlaid onto images of the surgical field anatomy acquired by a video camera.The superimposition of the digital and surgical field data requires calibration,tracking,and registration mechanisms to scale the 3D virtual images to the dimensions of the real-time images,and the combined video can be viewed by the surgeon using a variety of 3D display technologies[3].

Intraoperative CT scanning can make use of reference markers to circumvent the effects of movement during ventilation.By monitoring breathing cycles and labeling positional changes to track abdominal wall deformation,respiratory gating can be achieved[45,46].However,the amount of the liver surface area that is accessible for scanning is limited due to its proximity to the diaphragm,and this method requires a hybrid-operating suite with CT Instrumentation.It is also difficult to predict the intraoperative deformation of intrahepatic structures.Intraoperative ultrasound images can be compared to preoperative AR images to mitigate the effects of deformation[47,48].CT scanning can also be used periodically to obtain intraoperative imaging data for intrahepatic structures,albeit with the above-mentioned limitations[14].

Generating 3D images for AR from medical imaging data

3D images for AR are generated from conventional preoperative and intraoperative imaging data.These data usually consist of raw Digital Imaging and Communications in Medicine(DICOM)formatted data recorded during CT using slice thicknesses ranging from 0.63 to 2.50 mm.Thinner slice size increases the accuracy of reconstructed images.Data from CT are more suitable for 3D reconstruction than MRI data because MRI is lower resolution and acquires thicker slices(5 mm)than CT.However,CT-derived reconstructed images of the liver of patients with biliary tract dilation can be superimposed onto 3D images reconstructed from magnetic resonance cholangiopancreatography data to enhance AR-assisted navigation for hepatobiliary surgery.Studies have also reported the use of other imaging data for 3D image reconstruction,including X-rays and 2D or 3D B-US[18].Volume rendering and surface rendering are the methods of 3D reconstruction used in most AR systems.

Volume rendering

Images of the liver are generated from CT and MRI data(usually in DICOM format)through a process known as volume rendering(Fig.1(A)).Algorithms used for volume rendering include volume element projection,footprint table,and shear deformation.In volume rendering,the density values of all voxels in the CT or MRI images are replaced by opacity values and corresponding shades of the four color channels(red/green/blue/alpha or RGBA)or gray scale according to the RGBA transfer function,which de fines the RGBA value for all possible voxels.The actual volume of soft tissue structures cannot,however,be calculated using this method unless they are delineated prior to volume rendering.

A limited number of image-guided techniques are available for liver resection.Although imaging methods,such as B-ultrasonography(B-US)and computed tomography(CT),can be used to identify intrahepatic structures and tumor boundaries intraoperatively[13,14],the inability of these methods to provide real-time imaging data represents a critical disadvantage for surgical navigation.In addition to the discontinuity of intraoperative scanning,the 2D ultrasound images are largely insufficient for surgeons to construct an approximate 3D mental image of the intrahepatic structures.Liver deformation and movement of the abdominal contents during ventilation alters the shape and location of intrahepatic structures in real-time,further diminishing the usefulness of intraoperative scanning.Intraoperative AR provides in situ real-time information about the precise location of intrahepatic structures and overcomes the shortcomings of conventional surgical approaches.The use of AR for preoperative planning with the aid of 3D-printed models based on AR-generated 3D images can also advance the skill of liver surgeons.

Fig.1.Rendering methods for AR for liver surgery.(A)Volume rendering of the liver using 3D slices and raw DICOM data from enhanced CT imaging.Reconstituted structures include the liver,hepatic vasculature,biliary tree,and adjacent organs.(B)Schematic diagram of surface rendering for 3D liver reconstruction.

Surface rendering

In surface rendering,a mesh representing the delineated surface of the target is generated(generally a triangular mesh),which is subsequently used in 3D image reconstruction.Surface rendering is a more technically challenging process,but requires less data points than volume rendering methods.Surface rendering reconstructs 3D images of structures by a process known as mesh generation.It can be performed by generating and connecting 2D contour outlines from CT data to de fine the surface of a contrived 3D hollow structure without inner voxels(Fig.1(B))[12],or by extrapolating the 2D CT contour lines directly into a previously constructed 3D data field.Surface rendering generates much smaller image files than volume rendering,reducing the power requirements of the graphics card and computer processor relative to the requirements for volume rendering.Volume can be estimated based on images generated using surface rendering,but this method may be less accurate than calculations performed using volume rendering of delineated structures.One advantage of surface rendering is that it can be performed using partial data sets by using geometric elements as a substitute for original pixels,with a negligible overall effect resulting from the missing original data.

Organ delineation

Both automatic and manual segmentation methods have been developed for organ delineation,which use a variety of algorithms[21].Regardless of the method of segmentation used,manual delineation is always required for the liver and biliary structures because anatomical variation in hepatic morphology,blood vessels,and biliary ducts are common[2].Manually delineating contour lines require segmenting structures slice by slice to identify intrahepatic structures based on their density and morphology,and to simulate incision sites for preoperative planning.Commercial software can provide semiautomatic algorithms,and several companies also offer 3D modeling services using raw DICOM data.The Iqqa-Liver software(EDDA Technology,Princeton,New Jersey,USA)and the Hisense computer-assisted surgery system(Qingdao,China)have been used to generate 3D liver models at our institution.

Display technologies for AR

Intraoperative ultrasound and CT images can be superimposed onto target organs to discern internal structures[56].However,the 2D images are often unclear and the field of view may be too narrow to provide sufficient stereoscopic perception.The entire organ can be scanned intraoperatively,but the use of preoperative models can reduce operative time.Low-dose CT scans can be used in place of B-US for intraoperative registration,but the resolution of this method decreases with decreasing radiation dosage,which should be limited to 11–25 mGy[54].

Video-based AR display

A video-based AR display is commonly used in laparoscopic,robotic,and endoscopic procedures(Fig.2(A)and(B))[22].In a video-based display,the live surgical field and virtual 3D reconstructed images are shown simultaneously on one screen using an external video display or a head-mounted video display device(HMD).The advantages of this method are that all of the surgeons and assistants share the same visual perspective,and that it is easier to synchronize resolution and brightness to achieve smoother superimposition.The disadvantages of a video-based display are primarily related to the limitations of the 3D camera capturing the surgical field of view,as these often have a relatively low resolution.

Projection-based AR display

Projection-based AR display is achieved by projecting reconstructed images directly onto the surgical field of view(Fig.2(C))[23].Tracking and calibration of multiple anatomical structures are required to compensate for changes in the appearance of the projected images due to variations in body and organ surface curvature(Fig.2(G)and(H)),and to account for the surgeon’s angle of view and head position to avoid the misalignment of structures in the virtual image with the corresponding structures in the surgical field of view(Fig.2(I))[24].However,overlapping tracked structures can interfere with tracking and calibration [25],especially during open liver surgery because laparotomy exposes a relatively small surgical field.In addition,although the reconstructed images are 3D,the projected image is 2D,which may affect the surgeon’s depth perception.Consequently,this display method is most often used for procedures involving super ficial structures,such as blood vessels,or for visualizing the trocar pathway in preparation for laparoscopic surgery,and is not currently used for liver surgery.The development of 3D holographic projection technology might solve the problem of stereo overlap and diminished depth perception,thereby improving the utility of a projection-based display for intraoperative navigation.

See-through AR display

In see-through AR display technologies,surgeons view the operating field through a semitransparent,half-silvered mirror on which the reflection of the 3D reconstructed video images is superimposed.This display method allows surgeons to visualize actual organs and reconstructed images simultaneously,without diminishing tactile feedback[26],and provides a larger field of view than a video-based display.The semitransparent mirror may be mounted in an HMD or incorporated into eyeglasses(Fig.2(D))for see-through display[27],but HMDs are not often used for this purpose due to resolution and registration limitations and the physical burden caused by the weight of the HMD.The use of a stereoscope is also compatible with this display method by viewing the surgical field through the half-silvered mirror(Fig.2(E)).

Projection-based AR can depict representations of hepatic tumors,vessels,bile ducts,adjacent viscera and so on,by projecting images directly onto the skin of the abdomen to guide biopsy,radiofrequency ablation,percutaneous transhepatic cholangial drainage and angiography,which can minimize the operation time and avoid repetitive imaging scans.However,a projection-based display may alter the surgeon’s depth perception,and thereby complicate the localization of deeper structures[54,55].During surgery,the patient’s respiratory movements can displace the liver of more than 2 cm.Surgeons often tilt the operating table and induce pneumoperitoneum to gain better access to hepatic structures,but this can also cause registrations(Fig.5(A)and(B)).Organ deformation may be estimated based on a preoperative analysis of organ deformation under pneumoperitoneum conditions[45].Shekhar et al.[54]performed continuous low-dose CT and periodic MRI to provide information on organ shape and position intraoperatively.However,cumulative radiation dosage is a concern for intraoperative CT and the need for additional instrumentation in the operating suite is a consideration for both approaches.

Context-aware AR-assisted surgical navigation

Context-aware AR can indicate “diseased structures”and “target structures”during navigation to improve context awareness[34].It can also categorize the surgeon’s actions as “risky”by displaying combinations of words simultaneously,such as“blood vessels”and “cutting”,to warn against potential damage to nearby structures[34].By allowing the context-aware AR navigation system to identify organs and structures intraoperatively,the AR images can be displayed at key points only,rather than throughout the entire procedure,thereby reducing the level of distraction experienced by the surgeon[35].In simulated surgical experiments,this technology has been shown to be effective for guiding surgical procedures while minimizing interference with the surgeon’s vision.Improvements in identi fication,tracking,and registration for context awareness systems are needed for clinical applications in hepatobiliary surgery[36].

Fig.2.Basic principles of the AR display and the registration methods used in liver surgery.Video-based((A)and(B)),projection-based(C),and see-through((D)–(F))AR display methods.The method depicted in(A)requires 3D stereovision eyeglasses,and(B)and(D)depict HMD devices.No HMD is required for(C),whereas(E)represents microsurgery.An autostereoscopic see-through display is depicted in(F),which does not require stereo glasses or an HMD for 3D viewing.(G)–(I)Limitations of projectionbased AR display are primarily related to changes in the appearance of the 2D projected images due to changes in the curvature of the abdominal surface((G)and(H))and the surgeon’s viewing angle(I).(J)–(L)Autostereoscopic see-through AR display.(J)Light rays are projected onto the flat-screen display by transmission through the lens array.(K)Integral videography(IV)is used to generate the virtual 3D image,which is overlaid using a half-silvered mirror,through which the surgical field is viewed by the surgeon without the aid of stereovision eyeglasses or an HMD to view the overlaid images in 3D.(L)Autostereoscopic 3D display showing a CT-derived reconstructed 3D image superimposed on a 3D-printed model of the perihilar hepatic vasculature and biliary tree.

Calibration

Due to considerable anatomic variation in intrahepatic structures,real and virtual images must coincide precisely to ensure the effectiveness of AR-assisted hepatobiliary surgery.The navigation system needs to be calibrated periodically,including the zeroing of instruments,which involves registration using known markers or indicators.Calibration parameters include the positioning of the virtual camera and models,focal length and the zero point.Determining the coordinate transformation relationship between the virtual and real objects is critical to the superimposition of the 3D reconstructed images onto the intraoperative video images(Fig.3(A))[37].Known instrument parameters combined with reference markers can be used to make these calculations with the aid of optical surface scans[38].

Calibrations of the camera and spatial relationship between the camera lens and optical markers or electromagnetic coils are required for video-based AR display[39],and reference landmarks must be veri fied manually[40,41].In liver surgery,calibration of a see-through display can involve model setups or the arrangement of at least 4 non-coplanar reference markers,and might also require direct illumination of surface points for geometric extrapolation of the relationship between the real and virtual markers[42].Automatic calibration can reduce human error.It is possible to use a calibration probe to determine the relative locations between real and virtual objects and the surgeon’s pupils to prevent false overlap(Fig.3(B)),and the relative position of the probe to the tracker must be calibrated.Autostereoscopic 3D technology does not require calibration to the surgeon’s eyes[43,44].

Tracking

In previous studies,it has been shown that 3D imaging signi ficantly improved the liver anatomy perception of medical students[71,72],and an AR simulator was proposed to provide junior surgeons with a realistic learning environment for laparoscopic liver resection training[8].

Fig.3.Basic principles of AR calibration methods for liver surgery.(A)The coordinate transformation process for registration is depicted.(B)Apparent overlap of virtual images at different distances.A tracking error will occur if registration is not calibrated at a uniform distance.

Organ movement and deformation have a negative impact on the accuracy of the superimposed images.Interactive and automatic registration systems have been developed,which allow real-time intraoperative image acquisition and the production of the AR integrated 3D video of the surgical field to be repeated periodically to adapt to the changes in the orientation and spatial dimensions of the liver due to deformation and movement[16,17].However,these systems require hybrid operating rooms with CT or MRI instrumentation,and the use of these systems for liver surgery in humans,to the best of our knowledge,has not been reported.

Electromagnetic tracking holds promise for overcoming these obstacles.An electromagnetic field is generated across the patient and sensor coils are used as tracking markers and could potentially be adapted for use with a choledochoscope(Fig.4(E))[49].However,recent studies have shown that red–green–blue depth(RGBD)video tracking,and simultaneous localization and mapping(SLAM)can be used to improve the quality of intraoperative C-arm cone-beam CT images and correct for rigid patient movement by a global calibration of the intraoperative CT images with preoperative image data[50,51],which shows great promise for improving the contribution of intraoperative scanning to AR navigation.

Registration

Registration is the process by which corresponding data points in the virtual and real images are linked together into one coordinate system.Approaches can be divided into interactive and automatic registration.Registration methods include reference marker registration using opaque anatomical landmarks,image registration based on comparison of preoperative and intraoperative images,surface registration based on scanning,template registration based on comparison of template and intraoperative images,and deformable model registration for soft tissue analysis[52].

Fig.4.Basics of optical tracking methods in AR-assisted liver surgery.(A)–(C)Equipment for optical tracking.(A)Infrared cameras with an emitter probe.(B)Probe with tracking reflectors.(C)Probe placement for optical tracking using a 3D-printed model.(D)Example of a 2D coding pattern used as a tracking marker with an overlaid virtual 3D image of a tumor mass(yellow),the perihilar hepatic vasculature(veins in blue and arteries in red),and the biliary ducts(green).(E)Electromagnetic tracking system for choledochoscope navigation.

Interactive AR registration

In interactive AR,the real images and the preoperative reconstructed 3D images are overlaid and the position,scale and orientation of the virtual images are manually modi fied.Typical anatomical landmarks include the ribs,xiphoid process,liver boundaries,porta hepatis,gallbladder fossa and the inferior vena cava.Surgeons can adjust the location and size of the 3D images to correspond with those of the anatomical markers.An alternative method involves tracking these points intraoperatively using an emitter probe.Various computer algorithms have been reported for tracking-based interactive AR registration[12,53].

Automatic AR registration

To implement automatic registration,deformation factors must be considered,including intraoperative movement caused by the surgeon and the instruments,as these can also produce displacement.When acquiring intraoperative reconstructed images,the frequency of image updating must be compatible with the tempo of the surgical procedure,otherwise surgical progress can be delayed considerably,increasing the risk of intraoperative complications[54].It is recommended that preoperative models of the hepatic vasculature,biliary tree and the whole organ be checked during intraoperative imaging[55].

Projection-based AR registration

An autostereoscopic see-through 3D display can also be achieved by viewing the surgical field through a half-silvered glass plate that reflects autostereoscopic 3D video images generated by an integral videography device(Fig.2(F),(J)–(L))[28].Autostereoscopic 3D technology uses either a single cylindrical lens or an assembly of cylindrical lenses placed before a liquid crystal display screen[29,30].The reflected light enters the eyes from different angles,and the brain processes the stereo images to perceive depth of field[31],thereby maintaining motion parallax and binocular disparity without tracking eye movements[28].Autostereoscopic 3D technology provides a more realistic visual experience for the surgeon than that achieved using HMD-equipped AR display technologies[32],and this type of autostereoscopic display is also known as a full parallax 3D image display.The latest autostereoscopic 3D display technology provides improved stereo imaging over longer distances by combining microelectromechanical systems and communication and signal processing technology[33],but the use of this type of AR system for liver surgery has not been reported.

Video-based AR registratio n

Video-based AR registration has been used in robotic and laparoscopic liver surgery.Organ boundaries and other anatomic landmarks can be used for registration,and intraoperative ultrasound scanning can be used to locate blood vessels and biliary ducts within the liver parenchyma to guide registration.Although the accuracy of AR may be greater than the estimation of the surgeon,the precision of AR registration decreases as the area of the surgical field and the distance from the landmarks to the camera increases.The use of multidimensional ultrasound,ultrasound-CT multimodal image fusion(Fig.5(C)),low-dose CT and/or MRI to register images in the local surgical field can improve AR accuracy[14].However,when deformation occurs or the field of view changes,surgeons must reacquire the virtual images and repeat registration.

Display methods used for AR-assisted intraoperative navigation generally include a video-based display,see-through display,and a projection-based display,which are used to present the virtual reconstructed images overlaid on the surgical field in a 3D visual environment.The see-through display and the projection-based display present the images without a video monitors.Each method varies in the technology used,and has associated advantages and drawbacks.

Five hundred and ninety-seven patients were enrolled.We excluded from the statistical analysis five patients for incomplete data and 79 cirrhotic patients because of the negative effects of ascites and peripheral edema on nutritional parameters.

Another laparoscopic registration method involves comparing preoperative 3D models with intraoperative video images.Surface information is acquired using a real-time laparoscopic camera,which is used to adjust the preoperative model by implementing non rigid registration of the preoperative reconstructed structures with the real-time images of the organ surfaces[57].Although this method has been used in liver surgery,the wide range of organ surface morphologies acquired by an endoscopic camera confounds precise registration of extrahepatic blood vessels and bile ducts[57].Dumpuri et al.[58]sought to create parallax-like effects in reconstructed images based on surface data obtained using flexible models that simulated various liver deformation patterns.These data were used to build a sample library that could be used to predict liver displacement and deformation in video-based AR registration,including the corresponding changes in the shape and relative positions of the intrahepatic structures.

十点整，高潮看到台上的齐眉站起身，满面春风地望着会议厅入口处，鼓起掌来。高潮一边跟着鼓掌，一边顺齐眉的目光望过去。高潮看到入口处出现了一个五十岁开外的男人，身材中等，脑门光亮，戴着一副近视眼镜，正器宇轩昂笑容可掬地冲嘉宾席拱着手。在噼里啪啦的掌声中，那人走上主席台，坐在了正中间的位置上。他用眼角扫了一眼桌上的签到簿，问齐眉：朋友们都到齐了吧？

Fig.5.Projection-based and video-based image registration for AR.(A)and(B)when the reconstructed 3D images are displayed using a projection-based AR,a registration error can occur due to changes in the angle at which the images are displayed on the surgical field as a result of tilting the operating table to induce pneumoperitoneum.(C)In multimodal image-fusion technology,intraoperative ultrasonography video images can be compared with preoperative CT images using 3D image reconstruction and video registration to create intraoperative real-time virtual sonography imaging of the hepatic vasculature and tumors.Red shading indicates the location of the portal vein,and blue shading indicates the location of the trunk and primary branching of the hepatic vein.

Fig.6.Use of 3D-printed technology for guided liver surgery.Models may be whole or partial organs of variable texture and hardness that depict the patient’s healthy tissues and the pathological condition to facilitate the surgeon’s visualization of intrahepatic structures.The tumor mass is shown in yellow.Biliary ducts are shown in green.Veins and arteries are shown in blue and red,respectively.

Applications of AR for hepatobiliary surgery

3D images reconstructed from CT or MRI data can be used for both preoperative planning and intraoperative navigation.Virtual surgery planning systems,such as Virtual Surgical Planning(VPN,IRCAD),Iqqa-Liver,and MeVis(MeVis Distant Services,Bremen,Germany),use patient-speci fic modeling to generate animated simulations for intraoperative navigation based on preoperative images[59,60].These software platforms can also simulate resection,and calculate resection volumes.In addition,MeVis can predict intraoperative liver deformation,whereas VPN and Iqqa-Liver cannot consider deformation without the aid of additional software.At present,most software platforms mimic slicing by connecting cutting lines,rather than by slicing directly through the 3D virtual models,which can lead to discontinuity of the actual surgical incisions and the loss of stereoscopic perception.The 3D images generated during virtual surgical planning can be registered to the actual surgical field of view to judge the accuracy of the resection planes,and to guide the management of residual parenchyma along preoperatively designated boundaries[59].3D models of hepatobiliary structures produced using 3D printing technology can also be useful for preoperative planning(Fig.6)[61].

皮肤性痤疮的发病因素与个人卫生不佳、心理压力大、遗传、激素代谢紊乱、用药等因素相关,在多种因素作用下致使患者毛囊皮脂腺出现慢性炎症性皮肤病。痤疮不但会影响患者的容貌,还会对患者的心理造成一定影响,因此患者通常有着强烈的治疗意愿。激光在皮肤性痤疮治疗中取得了一定进展,经过治疗后,患者的丘疹、脓包、粉刺数量有所减少,皮损范围也明显缩小,在患者治疗期间加强综合护理,有助于纠正患者的不良生活习惯,促进患者更好的保护皮肤,从而进一步提高治疗效果,此外通过对患者的精心护理还有效提高了患者的护理满意度,对于提升我院整体护理质量具有重要作用。

A number of studies have reported the use of AR for open,laparoscopic and robotic liver resection[3,11,20].The use of AR in these different approaches for liver resection required innovations in registration and 3D display technologies.Here,we provide a summary of studies of AR-assisted liver surgeries representing important developments in the field,as well as the current status of experimental AR methods that have not yet been used in humans.

当代的潮州麦秆画创作，通过这类具有潮人情怀的人物画作品，例如：《汉学家饶宗颐》《陈伟南像》《李嘉诚像》等，传达出先辈所宣扬的潮人文化与潮人精神，并以此勉励后辈将潮人文化与潮人精神继续发扬光大。

AR for laparoscopic and robotic liver surgery

Patients with hepatocellular carcinoma(HCC)may bene fit from AR-assisted laparoscopic or robotic liver resection[11,20].The application of AR allows for more precise tumor localization,and can be used to delineate resection margins using video-based AR display.Buchs et al.[62]reported HCC resections in a 68-year-old man and an 82-year-old woman using a CAS-One guidance system(CAScination,Bern,Switzerland)and a Vicra instrument guidance system(Northern Digital,Waterloo,Canada),respectively.Registration time and safe margin widths were 91 s and 1 cm,respectively,for the first patient,and 136 s and 0.5 cm,respectively,for the second patient,with no complications in either case.

An AR-guided minimally invasive transthoracic liver resection was reported by Hallet et al.[10]in which a thin slice triphasic CT scan was performed to acquire DICOM files preoperatively.Images were reconstructed using the VPN program.During the 3D image reconstruction,a resection plan was devised in which resection margins were de fined relative to adjacent intrahepatic vasculature.For registration,visible landmarks were used to manually merge the images.The video camera display consisted of an exoscopic camera for the external portion for port placement combined with a laparoscopic camera for the intracorporeal portion.The virtual images and robotic scope video were overlaid using an MX 70 video mixer(Panasonic,Osaka,Japan).The 4-cm,well-differentiated HCC tumor in segment 8 was resected,with a total operative time of 270 min.and blood loss of approximately 300 mL.

In another study by the same research group,three patients underwent liver resection for benign hepatic tumor(segmentectomy V),HCC(segmentectomy V),and colorectal liver metastasis(segmentectomy VI)using a similar AR approach[11].Biomechanical predictions were made with segmented preoperative 3D reconstructed images of the viscera and skin after peritoneal insufflation with a constant intra-abdominal pressure of 12 mmHg.Surgeons registered alignment with visible landmarks intraoperatively.The time required to achieve AR registration ranged from 6 to 10 s,and hepatic pedicle clamping was not required.

AR for open liver surgery

Onda et al.[63]described a partial liver resection on a 60-yearold man with metastatic liver cancer and right hepatic lobectomy for cholangiocellular carcinoma in a 53-year-old man.A short rigid scope,two charge-coupled device cameras,a stereo display monitor and circularly polarized 3D glasses were used to create a 3D viewing environment similar to a stereoscopic laparoscope.The Analyze imaging software suite(Mayo Foundation,Rochester,NY,USA)was used for 3D image reconstruction and the liver and its vascular structures were delineated and rendered in color.Paired-point matching registration was performed by assessing 3 or more visceral landmarks,which required 1–2 min.for each registration period.Registration error was lower than that of image registration during open liver resection.

Okamoto et al.[26]used a similar approach with surface rendering and a video-based AR display in the treatment of three patients,which included a hepaticojejunostomy,a right liver resection with resection of the extrahepatic bile duct,and an HCC resection involving Couinaud segments 1 and 8.Registration was performed using points along the inferior vena cava,at the boundaries of the liver and within the hepatic hilum,with no instrument tracking.In the case requiring right liver resection,AR improved the accuracy of the resection line and shortened the operative time by eliminating the need for intraoperative cholangiography.Residual error between the practical resection line and the preoperative line was approximately 5 mm.The use of AR allowed preservation of the adjacent vasculature in the HCC resection.

（1）有效性原则。在制作、使用课件时，必须从实际出发，注重实效。工具使用不在于多而在于精，恰当运用多媒体，找准最佳作用点，有的放矢，才能起到画龙点睛的作用。

Pediatric tumor detection and resection can be difficult due to tumor adhesion and degeneration.Souzaki et al.[64]described a video-based AR navigation system for a patient with hepatoblastoma who underwent laparotomy for right lobectomy following chemotherapy without radiotherapy.An AR system was used to con firm the exact tumor margins and surgical borders.The operative time was 291 min.and the tumor size 4 cm.The resection of perihilar tumors ranks among the most challenging of liver surgeries.As part of our ongoing research,we have used a videobased AR display system for hilar cholangiocarcinoma resection during open surgery(Fig.7(A)and(B)).The utility of see-through and projection-based AR display for open liver surgery is limited,unless autostereoscopic 3D technology can be incorporated into the navigation system.

AR for liver endoscopic and natural ori fice transluminal endoscopic surgery

The use of AR technologies may reduce the loss of spatial orientation that commonly occurs in natural ori fice transluminal endoscopic surgery(NOTES)[65].The Aurora 3D electromagnetic tracking system(Northern Digital,Waterloo,Canada)consists of a tube containing miniature electromagnetic coils that is placed in the channel of a flexible endoscope to provide 3D imaging and localization of the endoscope in real-time.This technique can be used for navigation for endoscopic retrograde cholangiopancreatography and choledochoscopy[65].The localization of hepatoliths is also possible using 3D volumetric rendering and a see-through display during choledochoscopy[65].The bene fits of this display system can be enhanced by tracking the position of the scope lens using electromagnetic tracking,but the viewing angle of the lens can be difficult to determine.In addition,by using AR data analysis to evaluate tissue patterns,lesions that might otherwise escape visual detection can be identi fied[66].

AR for hepatic percutaneous procedures

Percutaneous thermal ablation is the most suitable for liver tumors with a diameter of 1–3 cm[67].However,in percutaneous thermal ablation,the needle tip is not always perfectly centered on an irregularly shaped tumor.Using traditional CT or digital subtraction angiography to guide punctures,practitioners may need to scan repeatedly to assess the location and depth of the tumor or bile duct,with reinsertion of the needle after each scan.By contrast,B-US imaging can be used for navigation without interrupting the procedure.However,B-US provides 2D images only and the ribs can interfere with B-US imaging of the liver.Nicolau et al.[68]have developed an AR guidance system for percutaneous hepatic thermal ablation in which CT was performed after marking the patient’s abdomen with radio-opaque rings.The 3D images were used to predict deformation of the abdomen and liver movement was predicted using respiratory gating techniques.Phantom trials using this method resulted in a margin of error＜2 mm and an accuracy during expiration＜4.5 mm.

AR-assisted ex situ liver resection and autotransplantation

Ex situ liver resection can be used to obtained R0 resection margins for otherwise nonresectable tumors(Fig.7(C)–(H)),and is suitable for cases of serious alveolar echinococcosis[69].Studies have reported the use of 3D reconstruction methods for assessing resectability and preoperative surgical planning for ex vivo liver resection[70].Although the use of AR-assisted navigation for ex situ resection has not been reported,the operating conditions should maximize the bene fits of AR-assisted navigation.Exposure of the surface of the liver during ex situ resection would facilitate scanning and tracking and reduce registration errors associated with organ deformation and displacement during ventilation.Scanning of the ex situ liver to visualize intrahepatic structures would also mitigate the effects of cholestasis and cirrhosis in cases of end-stage neoplasms and alveolar echinococcosis,which alter the deformation properties of the liver and confound efforts to estimate liver deformation in situ.

Discussion

Minimizing surgical trauma during liver surgery is paramount to reducing morbidity and mortality due to postoperative complications,but the need to obtain disease-free margins in liver resection is equally important for long-term clinical outcomes.The structure of the liver challenges the surgeon’s ability to visualize complex and variable intrahepatic anatomy to avoid injuring critical structures,which might result in excessive bleeding or biliary dysfunction.The level of surgical difficulty is compounded in minimally invasive approaches for liver surgery,which can further diminish 3D visualization and tactile proprioception.AR has been used in neurosurgery,otorhinolaryngology,vascular surgery,orthopedics,hepatobiliary surgery as well as urology and allows surgeons to visualize the liver anatomy preoperatively for surgical planning and intraoperatively for surgical navigation.

The tracking of intraoperative real-time cameras,including laparoscopic cameras,and the angle of the surgeon’s view is critical to ensure the accuracy of image registration.Commonly used tracking techniques include infrared,optical and electromagnetic methods.Several optical and electromagnetic tracking systems are commercially available,such as Certus Optotrak(NDI,Waterloo,Canada)and In finiTrack(Atracsys,Zug,Switzerland).Infrared tracking can reduce error to the submillimeter level.This type of system typically involves 2–4 infrared cameras and at least 4 reflectors,which serve as tracking markers.Fig.4(A)–(C)demonstrates the use of optical tracking to superimpose a CT-derived reconstructed 3D image of intrahepatic vascular and biliary structures onto a 3D-printed model.However,reflectors in close proximity may cause tracking interference,and obstacles may obscure the reflectors in the surgical field.Tracking reflectors can be replaced by an adhesive label with a 2D coding pattern(Fig.4(D)),such as that used by the MicronTracker system(ClaroNav,Toronto,Canada),but this can also be obscured intraoperatively by the surgeon’s hands and instruments.

感知风险是指消费者在使用无现金支付时，主观认为使用该支付方式带来的风险程度。消费者在无现金支付过程中可能面临病毒入侵手机、手机丢失、操作失误等带来钱财损失的财务风险，也可能面临身份信息、银行卡信息等个人信息被泄露甚至非法使用的隐私风险。消费者对风险的感知程度不一致，不同程度的感知风险，将会影响其对无现金支付的感知价值。

建章立制，将资金的分配、管理、使用等全过程的纳入制度的笼子，是做好乡镇财政资金监管的最根本保障，也是最有效的途径。乡镇财政部门应按照上级财政部门的要求，特别是财政部《关于切实加强乡镇财政资金监管工作的指导意见》（财预〔2012〕28号）等文件，进一步做好制度的梳理和完善工作，有计划、有步骤地开展乡镇财政资金监管的制度建设，逐步构架好“乡镇资金监管的制度牢笼”。通过制度的完善，把乡镇财政资金监管的责任层层分解到岗位、到人员，努力实现乡镇财政资金的全方位、无缝隙监管。

However,signi ficant improvements in all of the various components of AR are needed.Reconstruction techniques and registration methods need to be more precise and automated.Tracking techniques should be more accurate and provide real-time feedback.Display technologies require improved clarity and stronger,more natural three-dimensionality.Reducing the effects of intraoperative displacement and deformation of the liver is critical for improving the accuracy of registration.Surface topography scans may be useful for estimating deformation,but cannot accurately predict changes in the orientation of intrahepatic vasculature and biliary ducts.Although further research with biomechanical modeling may establish reliable methods for predicting changes in internal structures,intraoperative real-time ultrasound and CT imaging scans are the best currently available methods.The use of intraparenchymal reference markers for intraoperative CT scanning might bene fit efforts to more accurately estimate the orientation of intrahepatic structures under operating conditions[57].

捕食线虫真菌是指以营养菌丝特化形成的黏性菌丝、黏性分枝、黏性球、黏性网、非收缩环、收缩环及冠囊体等捕食器官来捕捉线虫的一类真菌，是自然界中线虫种群自然控制的重要因子〔1〕。由于在生防运用上表现出巨大的潜力，这类真菌已受到研究者的广泛关注。捕食线虫真菌作为一种兼性菌，既可营腐生生活，又可主动捕捉线虫来获取营养。在某些干扰因素的影响下，捕食线虫真菌获取营养的方式可能会发生改变。

Liao et al.have reported an integral videography technique with a semitransparent display that provides an intuitive AR environment for multiple observers by reproducing motion parallax in 3D images without the aid of an HMD or tracking devices[28,31].A related study has described a graphics processing unit and user interface that is capable of handling the high data burden of image reconstruction and registration,and provides efficient data transmission for smoother application of AR[73].These achievements represent important advancements for AR-assisted surgery,but the full potential of these developments for improving the precision of hepatobiliary surgical methods have not yet been achieved in liver surgery.The use of virtual images and video to improve medical education is also an important area of AR development.

Fig.7.AR-assisted surgical navigation for liver surgery.(A)and(B)laparotomy view of a patient with hilar cholangiocarcinoma using video-based AR navigation,which shows a surgeons view of the surgical field before(A)and after(B)tumor resection and hemihepatectomy.Note the 2D coding pattern affixed to the anterior surface of the right lobe,which was used for intraoperative registration and tracking.(C)–(H)Open surgery for alveolar echinococcosis.Ex situ liver resection(C)was performed,which revealed that Segments 2,3,and 4(D,E)were involved.Biliary ducts(green arrowheads),left hepatic vein(black arrowheads),left hepatic artery(red arrowheads),and regions of the portal vein(blue arrowheads)are indicated.(E)The reconstructed 3D images were superimposed on intraoperative digital images of the surgical field to simulate intraoperative AR overlay.(F):Autotransplantation was performed with reconstruction of the hepatic vein using a segment of jugular vein(black arrowheads).The junction of the bilioenteric anastomosis(green arrowhead),reconstructed portal vein(blue arrowhead),and left hepatic artery(red arrowhead)are indicated.(G)Preoperative enhanced CT cross-section showed the portal vein in Segment 3 and proliferation of the left hepatic lobe,and coronal section(H)showed right lobe invasion and partial left lobe invasion involving the inferior vena cava.

In conclusion,an increasing number of studies have demonstrated the use of AR technologies for preoperative assessment,surgical planning and intraoperative navigation in various types of liver surgery.With recent advancements,AR technologies have the potential for improving hepatobiliary surgical procedures.Future clinical studies are needed to evaluate the bene fits of AR for reducing postoperative morbidity and mortality and improving long-term clinical outcomes.Studies are also needed to investigate the fusion of multiple imaging modalities for intraoperative scanning,improve biomechanical liver modeling for predicting the location of hepatic ducts intraoperatively,and enhance image data processing and tracking technologies for image registration in order to increase the accuracy and utility of current AR systems.

Contributors

DJH proposed the study.TR and MLF performed the research and wrote the first draft.All authors contributed to the design and interpretation of the study and to further drafts.DJH is the guarantor.

Funding

This study was supported by grants from the Mission Plan Program of Beijing Municipal Administration of Hospitals(SML20152201),Beijing Municipal Administration of Hospitals Clinical Medicine Development of Special Funding(ZYLX201712),the National Natural Science Foundation of China(81427803),and Beijing Tsinghua Changgung Hospital Fund(12015C1039).

Ethical approval

Not needed.

Competing interest

No bene fits in any form have been received or will be received from a commercial party related directly or indirectly to the subject of this article.

首先来看新闻文本。尽管一则新闻报道指涉的是实在世界，但经过“把关人”(编辑、记者、传媒机构或任何信息发送者)对“材料集合”选择之后，其材料构成的成分已经是人为构建的符号组合，然后经过媒介化(文字或影像等)再现为显在的“述本”(公开发布的新闻报道)。这一“文本化”的过程实际上就是将所谓实在世界中的真相转化为文本所呈现出的真相。这正如博德里亚所说：“影像不再让人想象现实，因为它就是现实。”[注][法]博德里亚：《完美的罪行》，王为民译，北京：商务印书馆，2002年，第8页。也就是说，实在世界中的客观真相与文本所呈现的真相往往容易被混为一谈。

采用SPSS 17.0统计学软件对数据进行处理，计量资料以“±s”表示，采用t检验，计数资料采用x2检验，以P＜0.05为差异有统计学意义。

References

[1]Jarnagin WR,Fong Y,DeMatteo RP,Gonen M,Burke EC,Bodniewicz BS J,et al.Staging,resectability,and outcome in 225 patients with hilar cholangiocarcinoma.Ann Surg 2001;234:507–519.

[2]Hallet J,Gayet B,Tsung A,Wakabayashi G,Pessaux P.Systematic review of the use of pre-operative simulation and navigation for hepatectomy:current status and future perspectives.J Hepatobiliary Pancreat Sci 2015;22:353–362.

[3]Dong J,Yang S,Zeng J,Cai S,Ji W,Duan W,et al.Precision in liver surgery.Semin Liver Dis 2013;33:189–203.

[4]Peterhans M,vom Berg A,Dagon B,Inderbitzin D,Baur C,Candinas D,et al.A navigation system for open liver surgery:design,work flow and first clinical applications.Int J Med Robot 2011;7:7–16.

[5]Chen X,Xu L,Wang Y,Wang H,Wang F,Zeng X,et al.Development of a surgical navigation system based on augmented reality using an optical see-through head-mounted display.J Biomed Inform 2015;55:124–131.

[6]Kowalewski KF,Hendrie JD,Schmidt MW,Garrow CR,Bruckner T,Proctor T,et al.Development and validation of a sensor-and expert model-based training system for laparoscopic surgery:the iSurgeon.Surg Endosc 2017;31:2155–2165.

[7]Nickel F,Hendrie JD,Bruckner T,Kowalewski KF,Kenngott HG,Müller-Stich BP,et al.Successful learning of surgical liver anatomy in a computer-based teaching module.Int J Comput Assist Radiol Surg 2016;11:2295–2301.

[8]Strickland A,Fairhurst K,Lauder C,Hewett P,Maddern G.Development of an ex vivo simulated training model for laparoscopic liver resection.Surg Endosc 2011;25:1677–1682.

[9]Memeo R,de’Angelis N,de Blasi V,Cherkaoui Z,Brunetti O,Longo V,et al.Innovative surgical approaches for hepatocellular carcinoma.World J Hepatol 2016;8:591–596.

[10]Hallet J,Soler L,Diana M,Mutter D,Baumert TF,Habersetzer F,et al.Trans-thoracic minimally invasive liver resection guided by augmented reality.J Am Coll Surg 2015;220:e55–e60.

[11]Pessaux P,Diana M,Soler L,Piardi T,Mutter D,Marescaux J.Towards cybernetic surgery:robotic and augmented reality-assisted liver segmentectomy.Langenbecks Arch Surg 2015;400:381–385.

[12]Marescaux J,Clément JM,Tassetti V,Koehl C,Cotin S,Russier Y,et al.Virtual reality applied to hepatic surgery simulation:the next revolution.Ann Surg 1998;228:627–634.

[13]Torzilli G,Leoni P,Gendarini A,Calliada F,Olivari N,Makuuchi M.Ultrasound-guided liver resections for hepatocellular carcinoma.Hepatogastroenterology 2002;49:21–27.

[14]Kenngott HG,Wagner M,Gondan M,Nickel F,Nolden M,Fetzer A,et al.Real-time image guidance in laparoscopic liver surgery: first clinical experience with a guidance system based on intraoperative CT imaging.Surg Endosc 2014;28:933–940.

[15]Azuma R,Baillot Y,Behringer R,Feiner S,Julier S,MacIntyre B.Recent advances in augmented reality.Comput Graphics 2001;21:34–47.

[16]Haouchine N,Dequidt J,Berger MO,Cotin S.Deformation-based augmented reality for hepatic surgery.Stud Health Technol Inform 2013;184:182–188.

[17]Heizmann O,Zidowitz S,Bourquain H,Potthast S,Peitgen HO,Oertli D,et al.Assessment of intraoperative liver deformation during hepatic resection:prospective clinical study.World J Surg 2010;34:1887–1893.

[18]Beller S,Hünerbein M,Eulenstein S,Lange T,Schlag PM.Feasibility of navigated resection of liver tumors using multiplanar visualization of intraoperative 3-dimensional ultrasound data.Ann Surg 2007;246:288–294.

[19]Calhoun PS,Kuszyk BS,Heath DG,Carley JC,Fishman EK.Three-dimensional volume rendering of spiral CT data:theory and method.Radiographics 1999;19:745–764.

[20]Nicolau S,Soler L,Mutter D,Marescaux J.Augmented reality in laparoscopic surgical oncology.Surg Oncol 2011;20:189–201.

[21]Numminen K,Sipilä O,Mäkisalo H.Preoperative hepatic 3D models:virtual liver resection using three-dimensional imaging technique.Eur J Radiol 2005;56:179–184.

[22]Diana M,Marescaux J.Robotic surgery.Br J Surg 2015;102:e15–e28.

[23]Volonté F,Pugin F,Bucher P,Sugimoto M,Ratib O,Morel P.Augmented reality and image overlay navigation with OsiriX in laparoscopic and robotic surgery:not only a matter of fashion.J Hepatobiliary Pancreat Sci 2011;18:506–509.

[24]Shuhaiber JH.Augmented reality in surgery.Arch Surg 2004;139:170–174.

[25]Yasumuro Y,Imura M,Manabe Y,Oshiro O,Chihara K.Projection-based augmented reality with automated shape scanning.In:Proceedings of SPIE,5664.The International Society for Optical Engineering;2005.p.555–562.

[26]Okamoto T,Onda S,Matsumoto M,Gocho T,Futagawa Y,Fujioka S,et al.Utility of augmented reality system in hepatobiliary surgery.J Hepatobiliary Pancreat Sci 2013;20:249–253.

[27]Birkfellner W,Figl M,Huber K,Watzinger F,Wanschitz F,Hummel J,et al.A head-mounted operating binocular for augmented reality visualization in medicine-design and initial evaluation.IEEE Trans Med Imaging 2002;21:991–997.

[28]Liao H,Inomata T,Sakuma I,Dohi T.3-D augmented reality for MRI-guided surgery using integral videography autostereoscopic image overlay.IEEE Trans Biomed Eng 2010;57:1476–1486.

[29]Liao H,Hata N,Nakajima S,Iwahara M,Sakuma I,Dohi T.Surgical navigation by autostereoscopic image overlay of integral videography.IEEE Trans Inf Technol Biomed 2004;8:114–121.

[30]Liao H,Iwahara M,Hata N,Sakuma I,Dohi T,Koike T,et al.High-resolution integral videography autostereoscopic display using multi-projector.In:Proceedings of the ninth international display workshop,136;2002.p.A194.

[31]Liao H,Sakuma I,Dohi T.Development and evaluation of a medical autostereoscopic image integral videography for surgical navigation.In:Proceedings of the IEEE/ICME international conference on complex medical engineering-CME,2007;2007.p.213–218.

[32]Blackwell M,Nikou C,DiGioia AM,Kanade T.An image overlay system for medical data visualization.Med Image Anal 2000;4:67–72.

[33]Liao H.Super long viewing distance light homogeneous emitting three-dimensional display.Sci Rep 2015;5:9532.

[34]Kati´c D,Wekerle AL,Görtler J,Spengler P,Bodenstedt S,Röhl S,et al.Context-aware augmented reality in laparoscopic surgery.Comput Med Imaging Graph 2013;37:174–182.

[35]Lin HC,Shafran I,Yuh D,Hager GD.Towards automatic skill evaluation:detection and segmentation of robot-assisted surgical motions.Comput Aided Surg 2006;11:220–230.

[36]Rohl S,Bodenstedt S,Suwelack S,Dillmann R,Speidel S,Kenngott H,et al.Dense GPU-enhanced surface reconstruction from stereo endoscopic images for intraoperative registration.Med Phys 2012;39:1632–1645.

[37]Lapeer RJ,Jeffrey SJ,Dao JT,García GG,Chen M,Shickell SM,et al.Using a passive coordinate measurement arm for motion tracking of a rigid endoscope for augmented-reality image-guided surgery.Int J Med Robot 2014;10:65–77.

[38]Kellner F,Bolte B,Bruder G,Rautenberg U,Steinicke F,Lappe M,et al.Geometric calibration of head-mounted displays and its effects on distance estimation.IEEE Trans Vis Comput Graph 2012;18:589–596.

[39]Birkfellner W,Figl M,Huber K,Hummel J,Hanel RA,Homolka P,et al.Calibration of projection parameters in the varioscope AR,a head-mounted display for augmented-reality visualization in image-guided therapy.In:Proceedings of SPIE,4319;2001.p.471–480.

[40]Zhang Z.A flexible new technique for camera calibration.IEEE Trans Pattern Anal Mach Intell 2000;22:1330–1334.

[41]Ebisawa Y,Fukumoto K.Head-free,remote eye-gaze detection system based on pupil-corneal reflection method with easy calibration using two stereo-calibrated video cameras.IEEE Trans Biomed Eng 2013;60:2952–2960.

[42]Cash DM,Miga MI,Glasgow SC,Dawant BM,Clements LW,Cao Z,et al.Concepts and preliminary data toward the realization of image-guided liver surgery.J Gastrointest Surg 2007;11:844–859.

[43]Tang A,Zhou J,Owen C.Evaluation of calibration procedures for optical see-through head-mounted displays.In:Proceedings of the IEEE 2003 IEEE and ACM international symposium on mixed and augmented reality;2003.p.161–168.

[44]Gavaghan KA,Peterhans M,Oliveira-Santos T,Weber S.A portable image overlay projection device for computer-aided open liver surgery.IEEE Trans Biomed Eng 2011;58:1855–1864.

[45]Hostettler A,Nicolau SA,Rémond Y,Marescaux J,Soler L.A real-time predictive simulation of abdominal viscera positions during quiet free breathing.Prog Biophys Mol Biol 2010;103:169–184.

[46]Spinczyk D,Karwan A,Rudnicki J,Wróblewski T.Stereoscopic liver surface reconstruction.Wideochir Inne Tech Maloinwazyjne 2012;7:181–187.

[47]Lange T,Eulenstein S,Hünerbein M,Lamecker H,Schlag P-M.Augmenting intraoperative 3D ultrasound with preoperative models for navigation in liver surgery.In:Proceedings of the international conference medical image computing and computer-assisted intervention–MICCAI/Proceedings.DBLP 2004;2004.p.534–541.

[48]Lange T,Papenberg N,Heldmann S,Modersitzki J,Fischer B,Lamecker H,et al.3D ultrasound-CT registration of the liver using combined landmark-intensity information.Int J Comput Assist Radiol Surg 2009;4:79–88.

[49]Franz AM,Haidegger T,Birkfellner W,Cleary K,Peters TM,Maier-Hein L.Electromagnetic tracking in medicine-a review of technology,validation,and applications.IEEE Trans Med Imaging 2014;33:1702–1725.

[50]Fotouhi J,Fuerst B,Johnson A,Lee SC,Taylor R,Osgood G,et al.Pose-aware C-arm for automatic re-initialization of interventional 2D/3D image registration.Int J Comput Assist Radiol Surg 2017;12:1221–1230.

[51]Fotouhi J,Fuerst B,Wein W,Navab N.Can real-time RGBD enhance intraoperative cone-beam CT?Int J Comput Assist Radiol Surg 2017;12:1211–1219.

[52]Vemuri AS,Wu JC,Liu KC,Wu HS.Deformable three-dimensional model architecture for interactive augmented reality in minimally invasive surgery.Surg Endosc 2012;26:3655–3662.

[53]Scheuering M,Schenk A,Schneider A,Preim B,Greiner G.Intraoperative augmented reality for minimally invasive liver interventions.In:Proceedings of SPIE,5029.The International Society for Optical Engineering;2003.p.407–417.

[54]Shekhar R,Dandekar O,Bhat V,Philip M,Lei P,Godinez C,et al.Live augmented reality:a new visualization method for laparoscopic surgery using continuous volumetric computed tomography. Surg Endosc 2010;24:1976–1985.

[55]Nam WH,Kang DG,Lee D,Lee JY,Ra JB.Automatic registration between 3D intra-operative ultrasound and pre-operative CT images of the liver based on robust edge matching.Phys Med Biol 2012;57:69–91.

[56]Kang X,Azizian M,Wilson E,Wu K,Martin AD,Kane TD,et al.Stereoscopic augmented reality for laparoscopic surgery.Surg Endosc 2014;28:2227–2235.

[57]Rauth TP,Bao PQ,Galloway RL,Bieszczad J,Friets EM,Knaus DA,et al.Laparoscopic surface scanning and subsurface targeting:implications for image-guided laparoscopic liver surgery.Surgery 2007;142:207–214.

[58]Dumpuri P,Clements LW,Dawant BM,Miga MI.Model-updated image-guided liver surgery:preliminary results using surface characterization.Prog Biophys Mol Biol 2010;103:197–207.

[59]Radtke A,Sotiropoulos GC,Molmenti EP,Schroeder T,Peitgen HO,Frilling A,et al.Computer-assisted surgery planning for complex liver resections:when is it helpful?A single-center experience over an 8-year period.Ann Surg 2010;252:876–883.

[60]Soler L,Marescaux J.Patient-speci fic surgical simulation.World J Surg 2008;32:208–212.

[61]Igami T,Nakamura Y,Hirose T,Ebata T,Yokoyama Y,Sugawara G,et al.Application of a three-dimensional print of a liver in hepatectomy for small tumors invisible by intraoperative ultrasonography:preliminary experience.World J Surg 2014;38:3163–3166.

[62]Buchs NC,Volonte F,Pugin F,Toso C,Fusaglia M,Gavaghan K,et al.Augmented environments for the targeting of hepatic lesions during image-guided robotic liver surgery.J Surg Res 2013;184:825–831.

[63]Onda S,Okamoto T,Kanehira M,Fujioka S,Suzuki N,Hattori A,et al.Short rigid scope and stereo-scope designed speci fically for open abdominal navigation surgery:clinical application for hepatobiliary and pancreatic surgery.J Hepatobiliary Pancreat Sci 2013;20:448–453.

[64]Souzaki R,Ieiri S,Uemura M,Ohuchida K,Tomikawa M,Kinoshita Y,et al.An augmented reality navigation system for pediatric oncologic surgery based on preoperative CT and MRI images.J Pediatr Surg 2013;48:2479–2483.

[65]Soler L,Nicolau S,Hostettler A,Fasquel J-B,Agnus V,Charnoz A,et al.Virtual reality and augmented reality applied to endoscopic and NOTES procedures.In:Proceedings of the IFBME,25;2009.p.362–365.

[66]Mahmud N,Cohen J,Tsourides K,Berzin TM.Computer vision and augmented reality in gastrointestinal endoscopy.Gastroenterol Rep(Oxf)2015;3:179–184.

[67]Najmaei N,Mostafavi K,Shahbazi S,Azizian M.Image-guided techniques in renal and hepatic interventions.Int J Med Robot 2013;9:379–395.

[68]Nicolau SA,Pennec X,Soler L,Buy X,Gangi A,Ayache N,et al.An augmented reality system for liver thermal ablation:design and evaluation on clinical cases.Med Image Anal 2009;13:494–506.

[69]Pichlmayr R,Grosse H,Hauss J,Gubernatis G,Lamesch P,Bretschneider H.Technique and preliminary results of extracorporeal liver surgery(bench procedure)and of surgery on the in situ perfused liver.Br J Surg 1990;77:21–26.

[70]Mise Y,Tani K,Aoki T,Sakamoto Y,Hasegawa K,Sugawara Y,et al.Virtual liver resection:computer-assisted operation planning using a three-dimensional liver representation.J Hepatobiliary Pancreat Sci 2013;20:157–164.

[71]Beermann J,Tetzlaff R,Bruckner T,Schöebinger M,Müller-Stich BP,Gutt CN,et al.Three-dimensional visualisation improves understanding of surgical liver anatomy.Med Educ 2010;44:936–940.

[72]Müller-Stich BP,Löb N,Wald D,Bruckner T,Meinzer HP,Kadmon M,et al.Regular three-dimensional presentations improve in the identi fication of surgical liver anatomy–a randomized study.BMC Med Educ 2013;13:131.

[73]Herlambang N,Liao H,Matsumiya K,Masamune K,Dohi T.Interactive autotereoscopic medical image visualization system using GPU-accelerated integral videography direct volume rendering.Int J Comput Assist Radiol Surg 2008;3:110–111.