Towards a pancreatic surgery simulator based on model order reduction

Abstract

In this work a pancreatic surgery simulator is developed that provides the user with haptic feedback. The simulator is based on the use of model order reduction techniques, particularly Proper Generalized Decomposition methods. The just developed simulator presents some notable advancements with respect to existing works in the literature, such as the consideration of non-linear hyperelasticity for the constitutive modeling of soft tissues, an accurate description of contact between organs and momentum and energy conserving time integration schemes. Pancreas, liver, gall bladder, and duodenum are modeled in the simulator, thus providing with a very realistic and immersive perception to the user.

Background

It is now well known and scientifically demonstrated that the use of surgery simulators provides the practitioner with a fast method to developed the necessary skills [1]. And this is despite the well-known limitations that nowadays surgical simulators have [2, 3]. This is due to the complexity of the problem and the need for a feedback response at some 500 Hz to 1 kHz. Indeed, the problem is highly non-linear, due to both the constitutive modeling of soft living tissues, frequently considered as hyperelastic, and the non-linear phenomena taking place at the operating room: contact, friction, cutting, etc.

All these limitations make the development of surgery simulators a delicate task that has faced important difficulties in the last decades. It has not been until very recently that truly non-linear constitutive models have been developed [4–6]. In general, they are based on the employ of explicit finite element simulations, that allow for a fast resolution, element by element, of the equations of motion. However, these explicit algorithms are not unconditionally stable, and often lack of an appropriate energy conservation.

Recently, model order reduction techniques [7–9] have opened a different way of looking at real-time simulation. These techniques that, essentially, develop models with a minimal number of degrees of freedom, seem to be very well suited for the purpose of developing a real-time simulator [5, 10–12]. However, they are not free of limitations. In particular, projection-based (a posteriori) model order reduction techniques very often lack of efficiency in non-linear problems, where the complete system of equations needs to be rebuilt in order to perform consistent linearization, thus loosing all the pretended gain. Methods as the empirical interpolation method [13] or the coupling with Asymptotic Numerical Methods [14] aim at solving these deficiencies.

Proper Generalized Decomposition (PGD) techniques [15, 16], however, operate in a slightly different way. They operate by casting the problem in a parametric way, by considering every possible parameter of the problem (the position of application of the load, material parameters, etc.) as a new dimension in the phase space, end then solving the resulting high-dimensional problem off-line, once for life. A sort of response surface is then obtained that has been coined (in opposition to traditional response surface methodologies, that need for a well-developed campaign of computer experiments) as a computational vademecum [17].

This work is thus aimed at developing a prototype of real-time simulator for pancreatic surgery (very few examples exist, see [18]), able of providing an immersive response for surgery training and planning. This simulator should be able to run in standard laptops, without any supercomputing facility, thus being able to be used in the operating room (OR). In this framework, ta PGD-based surgery simulator here developed is composed by the necessary number of organ vademecums (depending on the particular type of surgery considered). These organ vademecums, that provide the system with each organ response to an applied load at any point of its surface, are then assembled together by considering their relative contact, giving finally a very realistic haptic sensation and immersive feeling.

Pancreatic cancer is one of the most severe illness, with the fourth rate of fatality among all types of cancer. Every year some 233,000 new cases are diagnosed worldwide. The pancreatic cancer characterization is extremely complex, for instance, with different types of cancer such as pancreatic cystic neoplasms and pancreatic neuroendocrine tumors among them, and often the preoperative procedures do not offer a conclusive diagnostic. Therefore, it is of utmost interest to have a system able to provide the surgeon with an augmented/virtual reality-based experience that could eventually help to make a diagnosis.

The complexity of the diagnosis is only one of the possible sources of difficulty. Under the name pancreatectomy (the surgical removal of all or part of the pancreas) different surgery procedures are encompassed. In pancreaticoduodenectomy, for instance, distal segment of the stomach, the first and second portions of the duodenum, the head of the pancreas, the bile duct, and the gallbladder are removed. This gives an idea of the difficulty of simulating such a surgery, see Fig. 1, where only the liver, duodenum, gall bladder and pancreas have been represented.

Fig. 1

Anatomy of the considered organs for the simulation of pancreatectomy. Source [19]

In the sequel we develop models for the aforementioned organs, considered as the most representative of the type of surgery at hand. Previously, and for the sake of completeness, we recall in “A review of PGD methods appliked to real-time surgery” the basics of the PGD method applied to real-time surgery. In “Perfomance” we analyze the performance of the resulting prototype. Finally, in “Conclusion” we draw some conclusions.

A review of PGD methods applied to real-time surgery

As mentioned before, the main novelty in PGD-based real-time simulators consist in developing a sort of a priori response surface, what we called a computational vademecum in [17]. Therefore, without any campaign of computer experiments, typical of response surface methodologies, PGD methods are able to provide in an off-line phase of the method with the expected response of the system in the form of a high-dimensional response surface or meta-model. This response surface is then able to run on-line at extremely high feedback rates.

One example of such a vademecum, for the simplest case, say quasi-static equilibrium for any (within a previously selected region) possible loading point, would provide us with an expression of the type

$$\begin{aligned} {\varvec{u}} = {\varvec{u}}({\varvec{x}}, {\varvec{s}}), \end{aligned}$$

(1)

i.e., a generalized solution of the displacement field of a solid undergoing a load at any possible point of its boundary, $\varvec{s}$. Such a high-dimensional response is found under the PGD rationale as a fight sum of separate functions, i.e.,

$$\begin{aligned} u^n_j({\varvec{x}},{\varvec{s}})=\sum _{k=1}^n X_j^k({\varvec{x}})\cdot Y_j^k({\varvec{s}}), \end{aligned}$$

(2)

where $u_j$ refers to the j-th component of the displacement vector, $j=1,2,3$ and functions $\varvec{X}^k$ and $\varvec{Y}^k$ represent the separated functions used to approximate the unknown field. To determine these functions, PGD methods proceed by first computing an admissible variation of $\varvec{u}$, by substituting them in the weak form of the problem, and subsequently linearizing it. This is usually accomplished by employing a greedy algorithm in which one sum is computed at a time, and within each sum, each function is determined by a fixed-point, alternate directions algorithm. The interested reader can consult more details of this approach in [2].

Should we need to consider (non-linear) dynamics, for instance, the PGD approach looks always for a sort of surface response, in this case in the form of an energy and momentum conserving integrator that provides the response of the system within a time increment $\triangle t$ in the form

$$\begin{aligned} {\varvec{u}}^{n+1}({\varvec{x}}, t+\triangle t,{ \varvec{u}}^t, {\varvec{v}}^t ) = {\varvec{u}}^n +{ \varvec{R}}({\varvec{x}})\circ {\varvec{S}}({\varvec{u}}^t)\circ {\varvec{T}}( {\varvec{v}}^t) \circ {\varvec{d}}(t), \end{aligned}$$

where the sought displacement field is now function (as obviously corresponds to an initial and boundary-value problem) of the initial conditions, seen as the converged displacement $\varvec{u}$ and velocity $\varvec{v}$ fields of the previous time increment. This approach has rendered excellent stability properties for linear and non-linear hyperelastodynamics [20].

Contact phenomena also play a crucial role in the simulation of surgery. In [21] a method based PGD was developed. In order to fully exploit the characteristics of PGD methods, one of the two candidate solids was embedded within a structured mesh in whose nodes the distance to the boundary field (i.e., a level set) was stored. The method then proceeds by checking that no boundary marker of the second solid crosses the zero-level set. Otherwise, a penalty force is applied at that point in order to prevent interpenetration. This very simple algorithm is able to run at haptic feedback rates without any problem.

Architecture of the simulator

Fig. 2

Architecture of the simulator

The PGD technique allows to exploit all the off-line effort of pre-computation and only post-process the result at extremely high feedback rates. Therefore, unlike previous examples of surgery simulators such as [22], for instance, there is no need to establish multiple threads in the simulation, nor establishing different feedback requirements for the different tasks in the simulator. In our approach, there is one single thread, see Fig. 2, and all the different procedures (contact detection among the virtual tool—assumed rigid for simplicity—and the organ(s), among the different organs themselves, displacement and strain field computation) run under the same constrain, that imposed by the haptic peripheral, a Geomagic Touch device [23] running the OpenHaptcis Toolkit in which all the system is developed.

Only rendering is accomplished under weaker requirements, usually in the order of some 30 Hz. In the rendering process the computation of nodal normal vectors at the deformed configuration of the solid is mandatory for an appropriate visualization. Even these normals could be pre-computed and stored in memory in the spirit of Eq. 1. However, our prototype showed that they can be computed in runtime without interfering with the main loop of haptic response. All these tests were made on a HP ProBook 6470b laptop (Intel Core i7, with 8 Gb DDR3 PC3-12800 SDRAM), see Fig. 3.

Fig. 3

Appearance of the developed simulator

Models for the different organs considered in the prototype

In this section we detail the performed work in the modeling of each organ, particularly each constitutive law employed for that purpose.

Liver

The liver is the biggest gland in the human body. It is connected to the diaphragm by the coronary ligament so it seems reasonable to assume it to be constrained at the posterior face by the rest of the organs, while the anterior face is accessible to the surgeon. The inferior vena cava travels along the posterior surface, and the liver is frequently assumed clamped a that location. The literature on the mechanical properties of the liver parenchyma is not very detailed. In [24] a Mooney–Rivlin and an Ogden models are compared to experimental results on deformations applied to a liver. No clear conclusion is obtained, however, given that no in vivo measurements could be performed. In view of that, we have assumed a simplified Kirchhoff-Saint Venant model, with Young’s modulus of 1.60 kPa, and a Poisson coefficient of 0.48, thus nearly incompressible [25]. This constitutes just a simplification that should be validated with the help of experienced surgeons, but remains valid as long as more complex models can be developed within the PGD techniques exposed before without any difficulty [26].

Fig. 4

Finite element mesh for the parenchyma of the liver

Fig. 5

First four spatial modes ${\varvec{X}}^k({\varvec{x}})$, $k=1, \ldots , 4$ (respectively, (a)–(d)) for the liver model

The finite element mesh of the liver, see Fig. 4 consists of 4349 nodes and 21,788 linear tetrahedral elements. The PGD modes obtained in the off-line procedure described before are shown in Fig. 5.

Gallbladder

The gallbladder is a pressurized vesicle attached to the liver and also connected to the duodenum. It contains the bile generated by the hepatocytes [27] provides the most comprehensive constitutive modeling of gallbladder walls. Gallbladder is composed by four different layers: adventitia, muscle, mucosa and epithelium. Of course, the muscle layer is responsible of expelling the bile right after consumption of foods or drinks.

By employing elastography experimental measurements [27], developed a model very closely resembling that of Gasser-Holzapfel-Ogden for arteries, previously employed in some of our previous works [10]. However, it is also stated that a linear elastic model that takes into account the heavy changes in gallbladder wall thickness produces almost the same results. Also a neo-Hookean model [28] has been found to accurately capture most of the deformation patterns of elastin. Therefore, we have adopted again, for simplicity and without loss in generality, a Kirchhoff-Saint Venant model, despite its well-known limitations.

Fig. 6

Finite element mesh for the gall bladder

The gallbladder is subjected, in normal conditions, to an internal pressure that has been estimated in [27] on 466.6 Pa. We have assumed a wall thickness on the order of 2.5 mm, and a Young’s modulus on the order of 1.15 kPa, with $\nu \,=\,$0.48, and thus nearly incompressible.

The finite element model, see Fig. 6, is composed by 4183 nodes and 17,540 linear tetrahedral elements Fig. 7.

Fig. 7

First four spatial modes ${\varvec{X}}^k({\varvec{x}})$, $k=1, \ldots , 4$ (respectively, (a)–(d)) for the gallbladder model

In principle the gallbladder has not been considered as attached to the liver, but to the duodenum only, since during the surgery procedure it needs to be detached from it, by appropriately simulating the scratching process done by the surgeon, more properly related to continuum damage mechanics than to cutting itself. This is currently one of our lines of research.

Pancreas and duodenum

In our model, pancreas and duodenum have been modeled as being attached to each other, see Fig. 8, since indeed they are, on one side, and most likely will be removed together, without detaching one from the other.

Very few papers deal with the constitutive modeling of pancreatic tissue, despite a few simulators exist, see for instance [29] for an example of a web-based navigation system. In [30] elastography is employed to determine the shear stiffness of the tissue, giving some 1.20 kPa at 40 Hz. In our simulations, an almost incompressible character (i.e., $\nu \,=\,$0.48) is assumed.

Fig. 8

Finite element mesh for the pancreas and duodenum

Fig. 9

First four spatial modes ${\varvec{X}}^k({\varvec{x}})$, $k=1, \ldots , 4$ (respectively, (a)–(d)) for the pancreas model

The two free opposite sides of the duodenum are considered clamped. In particular, the proximal one is indeed attached to the stomach, whose distal part is usually removed during this type of surgery. Therefore, in a more advanced version of the simulator, a model of the stomach should also be considered for completeness. In this proof of concept prototype, however, this simplification does not imply a loss of validity of the proposed methodology. In Fig. 9 the first four modes of the PGD approximation to the pancreatic vademecum are depicted.

Putting it all together: simulating contact

One of the most salient advantages of the procedure described before relies precisely in its modularity. Once a vademecum is computed for each organ, the resulting simulator integrates them all by simulating the contact between them, and their boundary conditions such as attachments to ligaments, tendons, blood vessels, etc.

The strong feedback requirements given by the haptic peripheral in terms of number of simulations per second and stability of the transmitted force in the haptic device, prevents us from using state-of-the-art FE frictional contact algorithms. Instead, we follow a simplified voxmap pointshell strategy [20] in which one of the solids in the model is considered the master and equipped with a distance field, see Fig. 10. This distance field is stored in memory at nodal positions in a lattice that surrounds each organ. In this case, the master is the gallbladder, since it occupies a central position among the simulated organs. The rest of the organs are marked with a pointshell, in our case composed by the nodes of the boundary (although a different set may be employed depending on the required precision).

Fig. 10

Distance field computed around the gallbladder model

One of the advantages of the vademecum approach is the possibility of computing a high-dimensional distance field

$$\begin{aligned} d=d(\varvec{x}, \varvec{s}) \end{aligned}$$

for every possible load position $\varvec{s}$, and to store it in memory to avoid the computation of distances in runtime. Since the deformed configuration of the solids is know beforehand, see Eq. (2) for every possible load position, orientation and modulus (which in this case could be a contact reaction), we can compute the distance field for each of these deformed configurations and store them in memory in the form of a sequence of separated vectors, as in Eq. (2).

Once the collision between the pointshell(s) and the zero-level iso-surface of the distance field vademecum is detected, a penalization force $\varvec{F}=-k_c\cdot d\cdot \varvec{n}$ is applied to both solids, so as to prevent penetration. Here, $\varvec{n}$ represents the normal to the surface in contact and $k_c$ a penalty parameter. Although this simple contact procedure is far from the state of the art in usual engineering practice, it must be kept in mind that the haptic feedback imposes an important bottleneck to the procedure: to avoid unphysical jumps in the response, producing a sequence of fast contact detections and loss of contact. This produces a very unphysical sensation that should be avoided. The proposed algorithm produces no unphysical sensation and no artificial jump in the haptic response could be felt. In fact, many existing simulators employ different threads for the simulation of contact and deformation, with considerably weaker requirements on the contact side see, for instance [22], among others. This is absolutely not necessary here, and a single thread is used in our architecture.

Our tests on the performance of the contact detection rendered feedback times ranging from 0.0007148 seconds as a lower bound, for the case in which no contact was detected and 0.0012811 seconds for situations in which more than 400 nodes were detected in contact with the zero-distance level set of the master surface. Both values are in good agreement with the feedback requirements imposed by the haptic device, while no artificial, unphysical jump was detected in the transmitted reaction forces, giving a smooth sensation of contact.

In Fig. 11 a test is made for the contact algorithm. In it, a time instant is shown in which a force applied in the gallbladder makes it to get into contact with the liver, whose displacement field is shown. On the contrary, duodenum and pancreas are at this time instant free of contact and therefore have been represented in wireframe to highlight it.

Fig. 11

Application of a force (indicated by an arrow) that makes liver and gallbladder to get into contact

Performance

Once the models have been developed and the PGD modes computed off-line, a realistic texturing is applied to the meshes so as to provide the user with an immersive sensation, see Fig. 12.

Fig. 12

Appearance of the simulator once the different organs have been textured

The number of modes necessary for a prescribed degree of accuracy in the feedback force can be determined either via heuristic approaches, with the help of experienced surgeons, or by employing error estimators on certain quantities of interest [31]. In general, the number of terms depends on the size of the models and the desired level of accuracy, but in our experience rarely exceeds some 50 modes. Keep in mind that the usual scientific computing levels of accuracy are rarely attained nor needed in this type of applications, where the variance of properties between patients is on the order of the mean.

In Fig. 13, for instance, a plot is shown of how the proposed PGD technique converges respect the reference solution given by a full FEM model of the pancreas. Note that the comparison between reduced models and FE reference ones is made on the basis of the same mesh size. Of course a more detailed FE mesh would imply a bigger memory usage for the reduced order model, but note that the CPU cost, and therefore the feedback speed, do not depend on the number of nodes of the mesh, but on the number of modes employed in the PGD approaximation, i.e., on n in Eq. (2), which remains roughly constant.

The applied load was divided into three load steps, following the technique presented in [26], where an explicit algorithm was developed. This is why the error seems to reach two intermediate plateaus. The evolution of the error in terms of the number of modes is perhaps better seen if we plot the error versus the number of modes employed per load step (i.e., the number of modes applied at the same time at load increment one, two and three). This is depicted in Fig. 14. In all these off-line computations, errors below 10 % were judged sufficient. Nonetheless, lower errors can be easily attained by adding more modes. It is well known that, at the limit, the full FE accuracy will be attained once the sufficient number of modes is considered (i.e., equivalent to the number of degrees of freedom of the full FE model).

Fig. 13

Convergence of the PGD approximation of a the pancreas vademecum towards the reference FEM solution for different number of modes

Fig. 14

Convergence of the PGD approximation of a the pancreas vademecum towards the reference FEM solution for different number of modes. Here, the number of modes represented at the abscisa is employed in all the three load increments

Under these circumstances, the just developed simulator provided feedback responses in the range of 600 Hz to 1 kHz. As mentioned before, no parallelization was needed and the prototype ran on a HP ProBook 6470b laptop (Intel Core i7, with 8 Gb DDR3 PC3-12800 SDRAM) without any appreciable jump in the force feedback, that remained very smooth throughout the simulation.

The developed method is so powerful that it can even be implemented on a html web page running javascript, see Fig. 15. In this case, without the requirements imposed by the haptic peripheral, javascript is able to provide the results with a feedback rate of more than 25 frames per second. This opens unprecedented opportunities for augmented learning strategies, for instance.

Fig. 15

Implementation on a web page running javascript

Simulation of surgical cutting in the context of PGD deserves some very specific comments. In [32] a method based on the combination of PGD and X-FEM technique was developed that provides very realistic sensations for the cutting procedure. The integration of this or other method in the framework of our simulator is currently one of our main efforts of research.

Conclusions

In this work a pancreatic surgery simulator has been developed by resorting to the concept of computational vademecum. A computational vademecum is a sort of computational response surface technique obtained without the need of a campaign of computer experiments. Instead, in an off-line phase, this response surface is obtained in the form of a finite sum of separable functions, typical of PGD methods.

The high-dimensional response thus obtained is exploited under severe real-time constraints in the on-line phase of the method. The method is able to generate quasi-static as well as dynamic approaches to the problem, including contact and surgical cutting. Other phenomena, such as scratching, are currently being studied and will hopefully be published elsewhere.

In any case, PGD methods provide a very appealing way of developing surgical simulators able to run is very simple platforms (typically, in a standard laptop), and even on smartphones or tablets [33]. Notably, it enables the possibility of developing surgical simulators including state of the art (usually, hyperelastic) constitutive laws and momentum and energy conserving, unconditionally stable, dynamical integrators.

The lack of suitable data for an accurate modeling of some soft living tissues (arteries are very well characterized, however, other tissues such as duodenum or pancreas very often lack of appropriate models other than linear elastic) remains, however a true limitation for the development of such a simulator. It should be compensated, for instance, by resorting to experienced surgeons that help engineers to increase the realism of the simulations.