Towards a pancreatic surgery simulator based on model order reduction

It is now well known and scientifically demonstrated that the use of surgery simulators provides the practitioner with a fast method to developed the necessary skills [1]. And this is despite the well-known limitations that nowadays surgical simulators have [2, 3]. This is due to the complexity of the problem and the need for a feedback response at some 500 Hz to 1 kHz. Indeed, the problem is highly non-linear, due to both the constitutive modeling of soft living tissues, frequently considered as hyperelastic, and the non-linear phenomena taking place at the operating room: contact, friction, cutting, etc.

All these limitations make the development of surgery simulators a delicate task that has faced important difficulties in the last decades. It has not been until very recently that truly non-linear constitutive models have been developed [4–6]. In general, they are based on the employ of explicit finite element simulations, that allow for a fast resolution, element by element, of the equations of motion. However, these explicit algorithms are not unconditionally stable, and often lack of an appropriate energy conservation.

Recently, model order reduction techniques [7–9] have opened a different way of looking at real-time simulation. These techniques that, essentially, develop models with a minimal number of degrees of freedom, seem to be very well suited for the purpose of developing a real-time simulator [5, 10–12]. However, they are not free of limitations. In particular, projection-based (a posteriori) model order reduction techniques very often lack of efficiency in non-linear problems, where the complete system of equations needs to be rebuilt in order to perform consistent linearization, thus loosing all the pretended gain. Methods as the empirical interpolation method [13] or the coupling with Asymptotic Numerical Methods [14] aim at solving these deficiencies.

Proper Generalized Decomposition (PGD) techniques [15, 16], however, operate in a slightly different way. They operate by casting the problem in a parametric way, by considering every possible parameter of the problem (the position of application of the load, material parameters, etc.) as a new dimension in the phase space, end then solving the resulting high-dimensional problem off-line, once for life. A sort of response surface is then obtained that has been coined (in opposition to traditional response surface methodologies, that need for a well-developed campaign of computer experiments) as a computational vademecum [17].

This work is thus aimed at developing a prototype of real-time simulator for pancreatic surgery (very few examples exist, see [18]), able of providing an immersive response for surgery training and planning. This simulator should be able to run in standard laptops, without any supercomputing facility, thus being able to be used in the operating room (OR). In this framework, ta PGD-based surgery simulator here developed is composed by the necessary number of organ vademecums (depending on the particular type of surgery considered). These organ vademecums, that provide the system with each organ response to an applied load at any point of its surface, are then assembled together by considering their relative contact, giving finally a very realistic haptic sensation and immersive feeling.

Pancreatic cancer is one of the most severe illness, with the fourth rate of fatality among all types of cancer. Every year some 233,000 new cases are diagnosed worldwide. The pancreatic cancer characterization is extremely complex, for instance, with different types of cancer such as pancreatic cystic neoplasms and pancreatic neuroendocrine tumors among them, and often the preoperative procedures do not offer a conclusive diagnostic. Therefore, it is of utmost interest to have a system able to provide the surgeon with an augmented/virtual reality-based experience that could eventually help to make a diagnosis.

The complexity of the diagnosis is only one of the possible sources of difficulty. Under the name pancreatectomy (the surgical removal of all or part of the pancreas) different surgery procedures are encompassed. In pancreaticoduodenectomy, for instance, distal segment of the stomach, the first and second portions of the duodenum, the head of the pancreas, the bile duct, and the gallbladder are removed. This gives an idea of the difficulty of simulating such a surgery, see Fig. 1, where only the liver, duodenum, gall bladder and pancreas have been represented.

In the sequel we develop models for the aforementioned organs, considered as the most representative of the type of surgery at hand. Previously, and for the sake of completeness, we recall in “A review of PGD methods appliked to real-time surgery” the basics of the PGD method applied to real-time surgery. In “Perfomance” we analyze the performance of the resulting prototype. Finally, in “Conclusion” we draw some conclusions.

As mentioned before, the main novelty in PGD-based real-time simulators consist in developing a sort of a priori response surface, what we called a computational vademecum in [17]. Therefore, without any campaign of computer experiments, typical of response surface methodologies, PGD methods are able to provide in an off-line phase of the method with the expected response of the system in the form of a high-dimensional response surface or meta-model. This response surface is then able to run on-line at extremely high feedback rates.

Contact phenomena also play a crucial role in the simulation of surgery. In [21] a method based PGD was developed. In order to fully exploit the characteristics of PGD methods, one of the two candidate solids was embedded within a structured mesh in whose nodes the distance to the boundary field (i.e., a level set) was stored. The method then proceeds by checking that no boundary marker of the second solid crosses the zero-level set. Otherwise, a penalty force is applied at that point in order to prevent interpenetration. This very simple algorithm is able to run at haptic feedback rates without any problem.

Architecture of the simulator

The PGD technique allows to exploit all the off-line effort of pre-computation and only post-process the result at extremely high feedback rates. Therefore, unlike previous examples of surgery simulators such as [22], for instance, there is no need to establish multiple threads in the simulation, nor establishing different feedback requirements for the different tasks in the simulator. In our approach, there is one single thread, see Fig. 2, and all the different procedures (contact detection among the virtual tool—assumed rigid for simplicity—and the organ(s), among the different organs themselves, displacement and strain field computation) run under the same constrain, that imposed by the haptic peripheral, a Geomagic Touch device [23] running the OpenHaptcis Toolkit in which all the system is developed.

Only rendering is accomplished under weaker requirements, usually in the order of some 30 Hz. In the rendering process the computation of nodal normal vectors at the deformed configuration of the solid is mandatory for an appropriate visualization. Even these normals could be pre-computed and stored in memory in the spirit of Eq. 1. However, our prototype showed that they can be computed in runtime without interfering with the main loop of haptic response. All these tests were made on a HP ProBook 6470b laptop (Intel Core i7, with 8 Gb DDR3 PC3-12800 SDRAM), see Fig. 3.

In this section we detail the performed work in the modeling of each organ, particularly each constitutive law employed for that purpose.

Liver

The liver is the biggest gland in the human body. It is connected to the diaphragm by the coronary ligament so it seems reasonable to assume it to be constrained at the posterior face by the rest of the organs, while the anterior face is accessible to the surgeon. The inferior vena cava travels along the posterior surface, and the liver is frequently assumed clamped a that location. The literature on the mechanical properties of the liver parenchyma is not very detailed. In [24] a Mooney–Rivlin and an Ogden models are compared to experimental results on deformations applied to a liver. No clear conclusion is obtained, however, given that no in vivo measurements could be performed. In view of that, we have assumed a simplified Kirchhoff-Saint Venant model, with Young’s modulus of 1.60 kPa, and a Poisson coefficient of 0.48, thus nearly incompressible [25]. This constitutes just a simplification that should be validated with the help of experienced surgeons, but remains valid as long as more complex models can be developed within the PGD techniques exposed before without any difficulty [26].

The finite element mesh of the liver, see Fig. 4 consists of 4349 nodes and 21,788 linear tetrahedral elements. The PGD modes obtained in the off-line procedure described before are shown in Fig. 5.

Gallbladder

The gallbladder is a pressurized vesicle attached to the liver and also connected to the duodenum. It contains the bile generated by the hepatocytes [27] provides the most comprehensive constitutive modeling of gallbladder walls. Gallbladder is composed by four different layers: adventitia, muscle, mucosa and epithelium. Of course, the muscle layer is responsible of expelling the bile right after consumption of foods or drinks.

By employing elastography experimental measurements [27], developed a model very closely resembling that of Gasser-Holzapfel-Ogden for arteries, previously employed in some of our previous works [10]. However, it is also stated that a linear elastic model that takes into account the heavy changes in gallbladder wall thickness produces almost the same results. Also a neo-Hookean model [28] has been found to accurately capture most of the deformation patterns of elastin. Therefore, we have adopted again, for simplicity and without loss in generality, a Kirchhoff-Saint Venant model, despite its well-known limitations.

The gallbladder is subjected, in normal conditions, to an internal pressure that has been estimated in [27] on 466.6 Pa. We have assumed a wall thickness on the order of 2.5 mm, and a Young’s modulus on the order of 1.15 kPa, with \(\nu \,=\,\)0.48, and thus nearly incompressible.

The finite element model, see Fig. 6, is composed by 4183 nodes and 17,540 linear tetrahedral elements Fig. 7.

In principle the gallbladder has not been considered as attached to the liver, but to the duodenum only, since during the surgery procedure it needs to be detached from it, by appropriately simulating the scratching process done by the surgeon, more properly related to continuum damage mechanics than to cutting itself. This is currently one of our lines of research.

Pancreas and duodenum

In our model, pancreas and duodenum have been modeled as being attached to each other, see Fig. 8, since indeed they are, on one side, and most likely will be removed together, without detaching one from the other.

Very few papers deal with the constitutive modeling of pancreatic tissue, despite a few simulators exist, see for instance [29] for an example of a web-based navigation system. In [30] elastography is employed to determine the shear stiffness of the tissue, giving some 1.20 kPa at 40 Hz. In our simulations, an almost incompressible character (i.e., \(\nu \,=\,\)0.48) is assumed.

The two free opposite sides of the duodenum are considered clamped. In particular, the proximal one is indeed attached to the stomach, whose distal part is usually removed during this type of surgery. Therefore, in a more advanced version of the simulator, a model of the stomach should also be considered for completeness. In this proof of concept prototype, however, this simplification does not imply a loss of validity of the proposed methodology. In Fig. 9 the first four modes of the PGD approximation to the pancreatic vademecum are depicted.

One of the most salient advantages of the procedure described before relies precisely in its modularity. Once a vademecum is computed for each organ, the resulting simulator integrates them all by simulating the contact between them, and their boundary conditions such as attachments to ligaments, tendons, blood vessels, etc.

The strong feedback requirements given by the haptic peripheral in terms of number of simulations per second and stability of the transmitted force in the haptic device, prevents us from using state-of-the-art FE frictional contact algorithms. Instead, we follow a simplified voxmap pointshell strategy [20] in which one of the solids in the model is considered the master and equipped with a distance field, see Fig. 10. This distance field is stored in memory at nodal positions in a lattice that surrounds each organ. In this case, the master is the gallbladder, since it occupies a central position among the simulated organs. The rest of the organs are marked with a pointshell, in our case composed by the nodes of the boundary (although a different set may be employed depending on the required precision).

Once the collision between the pointshell(s) and the zero-level iso-surface of the distance field vademecum is detected, a penalization force \(\varvec{F}=-k_c\cdot d\cdot \varvec{n}\) is applied to both solids, so as to prevent penetration. Here, \(\varvec{n}\) represents the normal to the surface in contact and \(k_c\) a penalty parameter. Although this simple contact procedure is far from the state of the art in usual engineering practice, it must be kept in mind that the haptic feedback imposes an important bottleneck to the procedure: to avoid unphysical jumps in the response, producing a sequence of fast contact detections and loss of contact. This produces a very unphysical sensation that should be avoided. The proposed algorithm produces no unphysical sensation and no artificial jump in the haptic response could be felt. In fact, many existing simulators employ different threads for the simulation of contact and deformation, with considerably weaker requirements on the contact side see, for instance [22], among others. This is absolutely not necessary here, and a single thread is used in our architecture.

Our tests on the performance of the contact detection rendered feedback times ranging from 0.0007148 seconds as a lower bound, for the case in which no contact was detected and 0.0012811 seconds for situations in which more than 400 nodes were detected in contact with the zero-distance level set of the master surface. Both values are in good agreement with the feedback requirements imposed by the haptic device, while no artificial, unphysical jump was detected in the transmitted reaction forces, giving a smooth sensation of contact.

In Fig. 11 a test is made for the contact algorithm. In it, a time instant is shown in which a force applied in the gallbladder makes it to get into contact with the liver, whose displacement field is shown. On the contrary, duodenum and pancreas are at this time instant free of contact and therefore have been represented in wireframe to highlight it.

Once the models have been developed and the PGD modes computed off-line, a realistic texturing is applied to the meshes so as to provide the user with an immersive sensation, see Fig. 12.

The number of modes necessary for a prescribed degree of accuracy in the feedback force can be determined either via heuristic approaches, with the help of experienced surgeons, or by employing error estimators on certain quantities of interest [31]. In general, the number of terms depends on the size of the models and the desired level of accuracy, but in our experience rarely exceeds some 50 modes. Keep in mind that the usual scientific computing levels of accuracy are rarely attained nor needed in this type of applications, where the variance of properties between patients is on the order of the mean.

In Fig. 13, for instance, a plot is shown of how the proposed PGD technique converges respect the reference solution given by a full FEM model of the pancreas. Note that the comparison between reduced models and FE reference ones is made on the basis of the same mesh size. Of course a more detailed FE mesh would imply a bigger memory usage for the reduced order model, but note that the CPU cost, and therefore the feedback speed, do not depend on the number of nodes of the mesh, but on the number of modes employed in the PGD approaximation, i.e., on n in Eq. (2), which remains roughly constant.

The applied load was divided into three load steps, following the technique presented in [26], where an explicit algorithm was developed. This is why the error seems to reach two intermediate plateaus. The evolution of the error in terms of the number of modes is perhaps better seen if we plot the error versus the number of modes employed per load step (i.e., the number of modes applied at the same time at load increment one, two and three). This is depicted in Fig. 14. In all these off-line computations, errors below 10 % were judged sufficient. Nonetheless, lower errors can be easily attained by adding more modes. It is well known that, at the limit, the full FE accuracy will be attained once the sufficient number of modes is considered (i.e., equivalent to the number of degrees of freedom of the full FE model).

Under these circumstances, the just developed simulator provided feedback responses in the range of 600 Hz to 1 kHz. As mentioned before, no parallelization was needed and the prototype ran on a HP ProBook 6470b laptop (Intel Core i7, with 8 Gb DDR3 PC3-12800 SDRAM) without any appreciable jump in the force feedback, that remained very smooth throughout the simulation.

The developed method is so powerful that it can even be implemented on a html web page running javascript, see Fig. 15. In this case, without the requirements imposed by the haptic peripheral, javascript is able to provide the results with a feedback rate of more than 25 frames per second. This opens unprecedented opportunities for augmented learning strategies, for instance.

Simulation of surgical cutting in the context of PGD deserves some very specific comments. In [32] a method based on the combination of PGD and X-FEM technique was developed that provides very realistic sensations for the cutting procedure. The integration of this or other method in the framework of our simulator is currently one of our main efforts of research.

In this work a pancreatic surgery simulator has been developed by resorting to the concept of computational vademecum. A computational vademecum is a sort of computational response surface technique obtained without the need of a campaign of computer experiments. Instead, in an off-line phase, this response surface is obtained in the form of a finite sum of separable functions, typical of PGD methods.

The high-dimensional response thus obtained is exploited under severe real-time constraints in the on-line phase of the method. The method is able to generate quasi-static as well as dynamic approaches to the problem, including contact and surgical cutting. Other phenomena, such as scratching, are currently being studied and will hopefully be published elsewhere.

In any case, PGD methods provide a very appealing way of developing surgical simulators able to run is very simple platforms (typically, in a standard laptop), and even on smartphones or tablets [33]. Notably, it enables the possibility of developing surgical simulators including state of the art (usually, hyperelastic) constitutive laws and momentum and energy conserving, unconditionally stable, dynamical integrators.

The lack of suitable data for an accurate modeling of some soft living tissues (arteries are very well characterized, however, other tissues such as duodenum or pancreas very often lack of appropriate models other than linear elastic) remains, however a true limitation for the development of such a simulator. It should be compensated, for instance, by resorting to experienced surgeons that help engineers to increase the realism of the simulations.