 Research Article
 Open Access
 Published:
Efficient solvers for timedependent problems: a review of IMEX, LATIN, PARAEXP and PARAREAL algorithms for heattype problems with potential use of approximate exponential integrators and reducedorder models
Advanced Modeling and Simulation in Engineering Sciencesvolume 3, Article number: 8 (2016)
Abstract
In this paper, we introduce and comment some recent efficient solvers for time dependent partial differential or ordinary differential problems, considering both linear and nonlinear cases. Here “efficient” may have different meanings, for instance a computational complexity which is better than standard time advance schemes as well as a strong parallel efficiency, especially parallelintime capability. More than a review, we will try to show the close links between the different approaches and set up a general framework that will allow us to combine the different approaches for more efficiency. As a complementary aspect, we will also discuss ways to include reducedorder models and fast approximate exponential integrators as fast global solvers. For developments and discussion, we will mainly focus on the heat equation, in both linear and nonlinear form.
Background
This paper deals with efficient numerical approaches to solve timedependent problems, possibly including parallelintime subdomain decomposition and making help of coarse reducedorder model solvers. As a typical problem of discussion, we will consider the classical heat equation: let \(\Omega \) be a bounded domain in \(\mathbb {R}^m\), \(m\in \{2,3\}\) with a Lipschitzcontinuous boundary. Let \(\kappa \) be a positive constant. Consider \(T>0\), \(u^0\in H^1_0(\Omega )\) and \(f\in L^2((0,T), L^2(\Omega ))\). The linear heat problem with \(u^0\) an initial value, homogeneous boundary conditions, for time t in the interval [0, T] reads
The problem (1) has a unique solution u in \(L^2((0,T),H^1_0(\Omega ))\). Semidiscretizing the problem (1) in space (method of lines) will classically lead to a highdimensional ordinary differential problem set in \(\mathbb {R}^d\) with generally large discrete dimension d. For simplicity, we will assume that the semidiscrete problem is written
with \(\varvec{u}^0\in \mathbb {R}^d\), \(\varvec{f}\in L^2((0,T),\mathbb {R}^d)\) and \(A\in \mathscr {M}_d(\mathbb {R})\) typically symmetric positive definite, with a sparse structure. In this paper we will also consider nonlinear versions of the heat problem with a thermal conductivity coefficient \(\kappa (u)\) depending on u itself. We will assume that there exists a constant \(\underline{\kappa }>0\) and a constant \(\overline{\kappa }> 0\) such that
The nonlinear heat problem reads
and we will assume that its semidiscretized form reads
with \(A(\varvec{u})\) sparse, symmetric positive definite for any \(\varvec{u}\), uniformly bounded. Let us now consider time discretization. Usually, time advance schemes for such kind of problems are chosen implicit or semiimplicit for stability purposes. As an exemple, the pure explicit Euler time advance scheme
where \(\varvec{u}^{n}\simeq \varvec{u}(.,t^n)\), \(t^{n+1}=t^n+\Delta t\) has a too restrictive numerical stability domain with typically \(\Delta t =O(h^2)\), h being representative of the space step size. Semiimplicit linear schemes in the form
show a far better stability domain but require the update of the stiffness matrix with the solution of a large linear sparse system at each time step. Finally, full implicit schemes
provide strong numerical stability but require fixedpoint (Newton or quasiNewton) algorithms for their numerical solution, what becomes computationally timeconsuming.
This paper gives an overview of recent alternative time advance schemes with interesting algorithmic features, including the possibility of parallel computations. First for linear problems, we will introduce the PARAEXP algorithm based on a superposition principle for achieving parallelintime computation. For nonlinear problems, the iterative LATIN method is a kind of splitting approach by alternating global linear solutions and local nonlinear projections. We will then discuss more general fixed point algorithms with a special focus on Newton and quasiNewton methods, separation of linear terms and nonlinear residuals in an implicitexplicit discretization strategy, then time subdomain decomposition and parallelintime computing involving coarse global and fine local propagators in the PARAREAL method.
The PARAEXP algorithm
Numerical methods allowing parallelization in the time direction have been thought since a long time (see Nievergelt [20] in 1964) and have known great developments particularly in the last decade because of today’s growing HPC platforms. Among timeparallel solvers, the PARAEXP algorithm introduced by Gander and Güttel [10] in 2013 is dedicated to linear ordinary differential problems, that is problems in the form
especially when \(\varvec{f}(t)\) is varying fastly in time. Problem (4) has a solution written in integral form thanks to the variationofconstant formula:
If we want to take advantage of (5) for deriving a numerical computational method, in particular we need a highorder quadrature formula of the integral term. If the \(\varvec{f}(t)\) are fast varying source terms, quadrature may become irrelevant from the accuracy point of view. Gander and Güttel rather propose to split the problem over p subdomains in time and use a superposition principle based on independent problems set onto different time domains:

1.
First, define a partitioning of the time domain [0,T] into p time subintervals \([T_{j1},T_j]\), \(j=1,...,p\), \(0=T_0<T_1<...<T_p=T\);

2.
For each \(j=1,...,p\), solve the initial zero value problem
$$\begin{aligned} {\dot{\varvec{v}}_j}(t) = A \varvec{v}_j(t) + \varvec{f}(t), \quad \varvec{v}_j(T_{j1})=0,\quad t\in [T_{j1}, T_j]; \end{aligned}$$(6) 
3.
For each \(j=1,...,p\), solve the homogeneous problem
$$\begin{aligned} {\dot{\varvec{w}}}_j(t) = A \varvec{w}_j(t), \quad \varvec{w}_j(T_{j1})=\varvec{v}_{j1}(T_{j1}), \quad t\in [T_{j1},T] \end{aligned}$$(7)(with the notation \(\varvec{v}_0(T_0):=\varvec{u}^0\)).
It is clear that by a superposition principle, on can synthesize a solution \(\varvec{u}\) of (4) by the summation formula
The PARAEXP algorithm is dedicated to parallel computing architectures, otherwise of course there is no benefit to execute it sequentially on one processor. It is remarkable to notice that good implementations of PARAEXP do not require any communication until the solution synthesis step, so the theoretical optimal efficiency is 1 before synthesis. Of course there is an issue of load balancing between processors because for a uniform time domain partitioning, some processors (especially the first one) are doing more work than others. The algorithm is graphically summarized in Fig. 1.
Another key of performance is the fast computation of the matrix exponentials. The solution of the homogeneous problem (7) in \([T_{j1},T]\) is
and thus has to be evaluated many times (at any time t in fact). There are many approaches to compute accurate approximate matrix exponentials as commented in [10]. A way is to search approximations into the Krylov subspace \(K^M\) of dimension M:
looking for a best approximation into the truncated series expansion. We will come back on this issue when reduced order models (ROM) will be introduced below.
Nonlinear problems: an implicitexplicit IMEX time advance scheme
Hereafter, we switch to the nonlinear case considering the nonlinear heat equation as reference example. Before going into iterative and parallel algorithms, let us first consider a variant implicitexplicit (IMEX) time advance scheme introduced by Filbet, Negulescu and Yang [8] in 2012. The idea is to consider an implicit linear diffusion term with an upper bound of the thermal conductivity, and explicit remainding terms at the right hand side:
with
As demonstrated in [8], let us show that the semidiscrete in time scheme is stable in the onedimensional case for a certain norm. Consider the homogeneous case \(f=0\) with homogeneous Neumann boundary conditions to simplify. We multiply (9) on \(u^{n+1}\) on the domain \(\Omega =(0,1)\), hence we have
Let us recall the PeterPaul inequality (extended Young’s inequality): for any nonnegative real numbers a and b, we have \(ab \le \varepsilon a^2/2 + b^2(2\varepsilon )\) for every \(\varepsilon >0\). Using the assumption that \(\kappa (u^n)\le \overline{\kappa }\), and applying PeterPaul’s inequality with \(a=\partial _x u^n\) we obtain
Therefore with the choice \(\varepsilon =\overline{\kappa }\), we have the weighted Sobolev norm decrease
This semidiscretization leads to a full discrete scheme in the form
with \(\varvec{g}(\varvec{u})\) in the form \(\varvec{g}(\varvec{u})= (\overline{A}A(\varvec{u}))\varvec{u}+ \varvec{f}\). What is interesting of course is that the matrix of the implicit part is constant, and thus has to be assembled and factorized once. Moreover the system is linear in the variable \(\varvec{u}^{n+1}\). Unfortunately, the PARAEXP algorithm cannot be applied directly here because the right hand side \(\varvec{g}(\varvec{u}^n)\) depends on the solution itself.
Iterative methods: the LATIN approach
A usual way to deal with nonlinear equations numerically is to use an iterative process within a fixed point algorithm. The LATIN (LArge Time INcremental) method pioneered by Ladevèze [15] and since then broadly used in computational structural Mechanics and material science (see [16] for a recent reference) solves timedependent problems (linear or nonlinear) according to a twostep iterative process. To separate the difficulties, equations are partitioned into two groups: (i) a group of equations being local in space and time, possibly nonlinear (representing equilibrium equations for example); (ii) a group of linear equations, possibly global in the spatial variable. Then adhoc spacetime approximations methods are used for the treatment of the global problem. Of course, spacetime local equations can be solved in parallel, what makes the LATIN method efficient and suitable for today’s HPC facilities. Let us emphasize that with LATIN, it is possible to solve hard nonlinear mechanics problems including thermodynamics irreversible problems (plasticity, friction as examples).
As an llustration, let us describe the LATIN method on the (rather simple) nonlinear heat problem:

1.
Initialization (\(k=0\)): let \(u_{(0)}\in L^2((0,T), H^1(\Omega ))\) an approximate solution (in space and time) of the nonlinear problem (it can be an approximate solution obtained with a coarse solver for example); compute \(\tilde{\kappa }_{(0)}=\kappa (u_{(0)})\);

2.
Iterate k, step 1 (global linear solution). Solve the linear problem
$$\begin{aligned} \partial _t u_{(k+1)}  \nabla \cdot (\tilde{\kappa }_{(k)} \nabla u_{(k+1)}) = f \end{aligned}$$with given initial and boundary conditions.

3.
Iterate k, step 2 (local projection over the admissible manifold). Compute
$$\begin{aligned} \tilde{\kappa }_{(k+1)} = \kappa (u_{(k+1)}). \end{aligned}$$ 
4.
Check convergence, \(k\leftarrow k+1\) if not and go to 2.
The step 1 performs a global (linear) evolution of the solution whereas a pointwise nonlinear projection on the equilibrium conductivity coefficients is done in step 2. We have a natural convergence indicator in terms of distance between the frozen conductivity \(\tilde{\kappa }\) and \(\kappa (u_{(k)})\):
In particular, if \(\kappa \) is Lipschitz continuous with Lipschitz constant L, then
Remark that step 2 can be performed in parallel (in time).
For better and faster convergence, one can imagine variant approaches using a relaxation approach: remark first that \(\kappa (u)\) is (formally) solution of the partial differential equation
If \(\kappa \) is a strictly convex function for example, then the second term at the right hand side is negative. One may consider the approximate (augmented) problem
where \(\varepsilon >0\) is a given relaxation time (assumed to be rather small). By this way, it is expected that \(\tilde{\kappa }\) evolves much closer toward the value \(\kappa (u)\). One can then derive an iterative process with again two steps (\(\hbox {linear solution} + \hbox {projection}\)) as in the LATIN method.
Newton and quasiNewton approaches
For the sake of simplicity, let us consider here the initial value problem with general autonomous system of ordinary differential equations
with \(\varvec{f}\) assumed to be differentiable and Lipschitz continuous, and initial condition \(\varvec{u}(0)=\varvec{u}^0\). The solution \(\varvec{u}\in L^2((0,T), \mathbb {R}^d)\) can be seen as the zero of a nonlinear operator \(\varvec{G}\),
The directional derivative of \(\varvec{G}\) at point \(\varvec{u}\) in the direction \(\varvec{v}\) is
Then the standard NewtonRaphson method applied to \(\varvec{G}\) reads for the kth iterate
that simplifies into
Hence, the NewtonRaphson method provides a sequence of linear problems (of unknown \(\varvec{u}_{(k+1)}\)) with variable coefficients and sources (depending on \(\varvec{f}\) and \(\varvec{u}_{(k)}\)).
Spectral structure of the linearized problem
Let us emphasize that, at a given k, the linear system has the expected spectral structure for approximate solutions near an equilibrium \({\overline{\varvec{u}}}\), that is \(\varvec{f}({\overline{\varvec{u}}})=0\). For \(\varvec{u}_{(k+1)}\) close to \({\overline{\varvec{u}}}\), we have
For \(\varvec{u}_{(k)}\) close to \({\overline{\varvec{u}}}\) and \(\varvec{f}\in \mathscr {C}^2\) we have
then
which is the expected linearized system.
QuasiNewton approach
As an additional approximation, a quasiNewton method will replace the Jacobian matrix \(D\varvec{f}(\varvec{u}_k)\) by an approximate one \(A_{(k)}\simeq D\varvec{f}(\varvec{u}_k)\), simpler to compute, thus giving the iterative process
If we are able to build some coarse approximation \(\varvec{g}\) of \(\varvec{f}\) such that the quasiNewton secant condition
is satisfied, we get the Jacobianfree quasiNewton iteration
or equivalently
In (16), \(\varvec{g}(\varvec{u}_{(k+1)})\) can be seen as a predictor term, whereas \((\varvec{f}(\varvec{u}_{(k)})  \varvec{g}(\varvec{u}_{(k)}))\) is a corrector term toward \(\varvec{f}\) depending on the iterate (k) only. By construction we retrieve the accuracy of \(\varvec{f}\) at convergence. A quasiNewton secant condition ensures superlinear convergence according to the Dennis and Moré theorem.
The PARAREAL method
The recent PARAREAL method, initially proposed by Lions et al. [17] in 2001, is nothing else but a parallelintime version of the quasiNewton method (16) above. In PARAREAL, the time domain is decomposed into p subdomains. Then we define a doubleindex sequence of approximate solutions \(\varvec{u}_{(k)}^j\), where k still denotes the current index of the iterative process and j is the number of the time subdomain \([T_{j1},T_j]\). In its regular current form (see [3, 4]), the PARAREAL algorithm is defined as follows:

1.
Define a partition in time \([T_{j1},T_{j}]\), \(0=T_0<T_1<...<T_p=T\);

2.
Define a cheap coarse propagator \(\mathscr {G}\) and a fine propagator \(\mathscr {F}\).

3.
Initialization (\(k=0\)): \(\varvec{u}_{(0)}^0 = \varvec{u}^0\), \(\varvec{u}_{(0)}^{j+1} = \mathscr {G}(\varvec{u}_{(0)}^j)\);

4.
Loop on the iterates k:
$$\begin{aligned} \varvec{u}_{(k+1)}^{j+1} = {\mathscr {G}(\varvec{u}_{(k+1)}^j)} \ + \ {\left( \mathscr {F}(\varvec{u}_{(k)}^j)  \mathscr {G}(\varvec{u}_{(k)}^j) \right) } \end{aligned}$$(17) 
5.
Check convergence, test the stop criterion.
The PARAREAL algorithm is graphically represented in the schematics of Fig. 2. From (17) and the graph dependency of Fig. 2, one can understand that each corrector term on time slice j
can be evaluated in parallel over the p processors. On the other hand the coarse propagator term \(\mathscr {G}(\varvec{u}_{(k+1)}^j)\) induces a persistent sequential part into the algorithm but it is expected to be evaluated quite fast. The tradeoff is to design a fast, “accurate enough” coarse propagator which does not affect the whole performance of the algorithm.
One can imagine different choices of coarse solvers: loworder accurate time advances schemes, simplified equations, simplified models, discretizations on coarser meshes, etc. Reference papers like Bal and Maday [4] and Baffico et al. [3] show general convergence theorems for nonlinear ordinary differential systems using coarse time integrators as coarse solvers. Gander and Hairer in [9] also show a superlinear convergence of the parareal algorithm.
Putting all together
Actually, there are different ways to mix the strategies seen so far. As an example, let us still consider the nonlinear heat equation with timevarying source term:
In the spirit of IMEX and LATIN, let us define the following iterative approach:
On the lefthand side of the equation, we have replaced the thermal conductivity \(\kappa (u)\) by some supremum as suggested by IMEX. In semidiscrete form, we get an equation in the form
We get a linear equation for the unknown \(\varvec{u}_{(k+1)}\) with constant coefficient matrix \(\overline{A}\), and the right hand side only depends on time through f(t) and \(\varvec{u}_{(k)}(t)\). Then the PARAEXP algorithm can be applied at each iterative k. The remaining nonlinear operations like (19) and the assembling of \(\varvec{r}_{(k)}\) can be done in parallel (in time). In conclusion, we have replaced a nonlinear problem by a sequence of linear problems where some nonlinear evaluations have been sent into the right hand side, and so can be computed in parallel.
The Newton method to handle nonlinear terms with ROMs of dynamical systems
Reducedorder modeling is a general methodology to determine the principal information of a general highdimensional problem and then reduce the problem, for example by projection. Reduction is generally possible when the MKolmogorov width
into an admissible close set U of a Banach space V is rather small for a rather small integer M (the dimension of the approximate space). One of the main motivations to do that is to strongly reduce the computational cost for the numerical solution. Even if there are recent advances in nonlinear reduced order modeling, in particular with the empirical interpolation method (EIM) proposed by Maday et al. [18], or discrete empirical interpolation method (DEIM) by Chaturantabut and Sorensen [5], there are still some issues and open problems for nonlinear timedependent problems. Dealing with general nonlinear terms and reducedorder modeling for dynamical systems may be a difficult task, because:

reducedorder models are expected to reproduce the stability of the system (for instance in the sense of Lyapunov, see [14] on this subject);

the local dynamics has to be reproduced, at least “at first order”, involving a compatibility of the spectral properties between full and reduced systems;

the area visited by the trajectories into the statespace may be defined over a nonlinear manifold rather than in a linear subspace. Thus nonlinear dimensionality reduction methods would be better candidates for reduction.
Balanced truncation strategy [1, 22] for example is a tradeoff in the reduction process to provide sufficient accuracy for controllability and observability of dynamical systems. However the theory mainly deals with linear timeinvariant (LTI) systems.
For timedependent problems, one can adopt a greedy incremental strategy during time by adapting/enriching the lowdimensional subspace when the principal components are changing during time. But the price to pay is to online evaluate some (highdimensional) nonlinear terms to control the error, what can be a penalizing factor of performance. If there is no other choice, parallelintime computing once again appears to be a complementary tool to keep global performance of the method.
Newton method and Galerkin projection method
Let us go back to the Newton method (13) that we rewrite here again
Let us consider a Galerkin approximation into the linear vector space
and assume that \((\varvec{w}^\ell _{(k)},\varvec{w}^m_{(k)})=\delta _{\ell m}\), \(1\le \ell ,m\le M\). We are looking for an approximate rankM solution \(\varvec{u}_{k+1}^M(t)\) in \(V^M_{(k)}\), i.e.
for some real coefficients \(a^m_{(k+1)}(t)\), \(1\le m\le M\) at time t. In order to get a reduced system, the Eq. (13) is projected onto the vector space \(V_{(k)}^M\). Multipling (13) by any test function \(\varvec{v}^M\in V_{(k)}^M\), we look for a loworder solution \(\varvec{u}_{(k+1)}^M(t)\) in the form (21) such that
Taking \(\varvec{v}^M=\varvec{w}^m_{(k)}\), \(1\le m\le M\), by orthogonality of the eigenvectors we get
In vector form, one obtains a reduced system in the form
with \(\varvec{a}_{(k+1)}^M(t)=(a_{(k+1)}^m(t))_m\), \((\tilde{A}_{(k)}^M)_{\ell m}(t)=(D\varvec{f}(\varvec{u}_{(k)}(t))\varvec{w}^\ell _{(k)},\varvec{w}^m_{(k)})\) and \((\varvec{r}_{(k)}^M(t))_m=(\varvec{f}(\varvec{u}_{(k)}(t))D\varvec{f}(\varvec{u}_{(k)}(t))\varvec{u}_{(k)},\varvec{w}^m_{(k)})\). Remark that when the initial system is linear, i.e. \(\varvec{f}(\varvec{u})=A \varvec{u}\), we retrieve the classical Galerkin projection over the space \(V^M\):
with a constant matrix \(\tilde{A}_{(k)}^M\), \((\tilde{A}_{(k)}^M)_{\ell m} = (A\varvec{w}^\ell _{(k)},\varvec{w}^m_{(k)})\). The assembling of both \(\tilde{A}_{(k)}^M(t)\) and \(\varvec{r}_{(k)}^M(t)\) requires highdimensional operations, but, fortunately, one can do this task in parallel (in time). Thus, one can nonetheless expect to get rather high performance. To summarize, at this stage of analysis, the algorithm of reducedorder modeling is the following:

1.
(initialization). Use a coarse solver and compute \(\varvec{u}_{(0)}\). Loop over (k):

2.
Compute M principal components \(\varvec{w}_{(k)}^m\), \(m=1,\ldots ,M\) or a suitable reduced basis from the knowledge of \(\varvec{u}_{(k)}\).

3.
Assemble and compute in parallel \(\tilde{A}_{(k)}^M(t)\) and \(\varvec{r}_{(k)}^M(t)\) at all the discrete times.

4.
Solve the linear problem
$$\begin{aligned}&{\dot{ \varvec{a}}}_{(k+1)}^M = \tilde{A}_{(k)}^M(t)\,\varvec{a}_{(k+1)}^M + \varvec{r}_{(k)}^M(t), \quad t\in (0,T]\\&\varvec{a}_{(k+1)}^M(0) = \varvec{a}^0_{(k+1)}\in \mathbb {R}^M, \end{aligned}$$and compute
$$\begin{aligned} \varvec{u}_{(k+1)}^M(t) = \sum _{m=1}^M a^m_{(k+1)}(t)\, \varvec{w}^m_{(k)}. \end{aligned}$$ 
5.
Test convergence after iterate k.
Remark 1
For the computation of the basis functions \(\varvec{w}_{(k)}^m\), one can of course use Proper Orthogonal Decomposition (POD) [22] or any other dimensionality reduction method. The update the reduced basis may also be done by incrementing the basis set within an adaptive learning algorithm.
Remark 2
In the step 3, it is assumed that both \(\tilde{A}_{(k)}(t)\) and \(\varvec{r}_{(k)}(t)\) have to be assembled and computed at all the discrete times. Of course, that may appear too penalizing for achieving high performance. Actually, one can consider additional reduction strategies for approximating both Jacobian matrix and right hand sides. This will be the aim of the following “Discussion” section.
Discussion about further reduction
There are many options to improve the whole numerical complexity of the algorithm using some additional approximations or reduction strategies.
Freezing up the Jacobian matrices
Let us go back to the Newton method
where the correction term \(D\varvec{f}(\varvec{u}_{(k)})(\varvec{u}_{(k+1)}\varvec{u}_{(k)})\) ensures quadratic convergence when it is converges. As already discussed in “Newton and quasiNewton approaches”, one can approximate the Jacobian matrix by some approximation \(A_{(k)}(t)\) which is cheaper to evaluate, leading to the quasiNewton approach
The matrices \(A_{(k)}\) still depend on time t a priori. But one could consider frozen approximates Jacobian matrices \(A_{(k)}^j\) of time slices \([T_j,T_{j+1}]\), further inviting for a parallelintime strategy.
Adding coarse models
If we do not want to worry about Jacobian matrices, then the other option is to consider a coarse model \(\varvec{g}\) of \(\varvec{f}\) as mentioned in “Newton and quasiNewton approaches” section. In this case, the quasiNewton iteration reads
In order to achieve an efficient reducedorder model, one have now to deal with the nonlinear term \(\varvec{g}(\varvec{u}_{(k+1)})\). An efficient and tractable way to proceed is to use an empirical interpolation method (EIM, [18]) for that. In that case, we can even make \(\varvec{g}\) depend on (k), according to some adaptive learning process (greedy algorithm, inflating basis, etc). Remark finally that the iterative process can once again be set up into a parallelintime framework following ideas from the PARAREAL algorithm.
Achieving dimensionality reduction for \(\varvec{f}\)
If possible, one can also use a reducedorder approximation for \(\varvec{f}\). If the iterative algorithm is expected to converge towards a solution that has the same order of accuracy than the original one, one have to consider an accurate reducedorder model for \(\varvec{f}\). Once the empirical interpolation method may help us for that. However, if a globalintime reduction strategy is considered, it is possible that the dimension M of the loworder vector space becomes too large, leading to a degradation of the whole performance.
An alternative approach would be to consider a family of localintime empirical interpolation methods for \(\varvec{f}\). In this case, we should also consider local models \(\varvec{f}_{(k)}^j\) available in the time slice \([T_j, T_{j+1}[\) which can also be updated at each k from a learning process.
Approximate exponential integrators
In order to make the PARAEXP algorithm globally efficient, it is essential to compute fast and accurate approximate exponential integrators. In the case of the linear heat equation, we have to compute the exponential of a large scale, symmetric sparse matrix A. More precisely, for the the problem \({\dot{\varvec{u}}} = A\varvec{u}\) with initial data \(\varvec{u}(0)=\varvec{u}^0\), we have to compute the solution \(\varvec{u}(t)=\exp (tA)\varvec{u}^0\) for any \(t\in [0,T]\).
As mentioned in [9], there are numerous techniques to determine accurate exponential matrices. Among then, one can for example mention Padé approximants, exponentially fitted integration methods, or approximations based on projections over Krylov subspaces
through Arnoldi orthogonalization iterations [12]. Actually the KrylovGalerkin projection can be seen as a reducedorder technique, with a suitable reduced basis that fits action of matrix exponentials. But of course there are other choices of suitable basis functions like the first eigenvectors \(\phi ^m\) of A:
For A symmetric positive definite with eigenvalues arranged in increasing order, that is \(0<\lambda ^1\le \lambda ^2 \le \ldots \lambda ^M \le \ldots \), it is natural to consider from the approximation error point of view the M first eigenvectors of A as vectors spanning the reduced approximation subspace. We will denote \(\tilde{A}\) the projection of A on this discrete subspace and of course we have \(\text {rank}(A)=M\). Considering the iterative approach of linear problems
by superposition principle, one can first consider the loworder homogeneous problem
for which we have an efficient loworder exponential solution, and on the other side the highdimensional problem with zero initial value
then \(\varvec{u}_{(k+1)}(t)=\varvec{v}_{(k+1)}(t)+\varvec{w}_{(k+1)}(t)\). In the spirit of the PARAEXP algorithm, one can set up the superposition principle within a parallelintime time decomposition to deal separately with loworder homogeneous exponential solution and highdimensional inhomogeneous problems.
Closing discussion
From this review on efficient timeadvance solvers including IMEX, LATIN, PARAEXP and PARAREAL algorithms, we try to show the different ways and tracks to deal with largescale dynamical systems, linear and/or nonlinear terms. For the sake of an easy discussion, we have taken the example of the heat equation (linear or nonlinear). We are aware that this may be too restrictive and nonlinear computational mechanics including for example thermodynamics irreversible problems need more efforts and technical developments. Among the methods discussed above, some of them have been designed to address these problems. This is the case for the LATIN approach for example.
Time parallelization appears to be a promising key element of speedup. For problems with a small Kolmogorov width, reducedorder modeling may be a supplementary methodology to accelerate the whole time advance solution. For numerous reasons, it is interesting to cast a nonlinear problem into a sequence of linear problems within an iterative process. Linear problems are easier to deal with, and there are dedicated tools like the parallelintime PARAEXP method. On the other hand, an iterative process allows for achieving multifidelity adaptive solvers, using incremental, greedy or learning algorithms. Of course, we have to keep in mind that iterative methods may not converge. So in the design process of the numerical approach, one has to answer to the following questions: is the whole iterative process stable, is it possible to prove the convergence ? If the method is convergent, what is the rate of convergence ? Is it possible to accelerate the convergence ? At convergence, is it sure that the iterative algorithm converges to the solution obtaines with the accuracy we paid at the finest level ? For parallel algorithms, what is the effective speedup ?
Last but not least, managing multifidelity models and multilevel reducedorder models as well as parallelintime algorithms and learning algorithms implemented on distributed memory computer architecture necessarily require data management efforts and smart software engineering.
Conclusions
The first aim of this paper is to review different efficient timeadvance solvers (including IMEX, PARAEXP, LATIN, PARAREAL) and show connections between them. We also try to show the links with quasiNewton approaches and relaxation/projection methods to deal with nonlinear terms. Parallelintime algorithms appear to be a complementary and promising framework for the fast solution of timedependent problems. Finally, reducedorder models (PODbased, principal eigenstructure, a priori reduced bases, ...) can be possibly included to achieve better performance. In a future paper, we will achieve numerical experiments on different hybrid approaches.
References
 1.
Antoulas AC. An overview of approximation methods for largescale dynamical systems. Annu Rev Control. 2005;29:181–90.
 2.
Audouze C, De Vuyst F, Nair PB. Nonintrusive reducedorder modeling of parametrized timedependent partial differential equations. Numeri Methods Partial Differen Equ. 2013;29(5):1587–628.
 3.
Baffico L, Bernard S, Maday Y, Turinici G, Zrah G. Parallelintime molecular dynamics simulations. Phys Rev E. 2002;66(5):057701.
 4.
Bal G, Maday Y. A “parareal” time discretization for nonlinear PDEs with application to the pricing of an American put. Recent developments in domain decomposition methods. Berlin: Springer; 2002. p. 189–202.
 5.
Chaturantabut S, Sorensen DC. Nonlinear model reduction via discrete empirical interpolation. SIAM J Sci Comput. 2010;32(5):2737–64.
 6.
Chinesta F, Ammar A, Lemarchand F, Beauchène P, Boust F. Parallel time integration and high resolution homogenization. CMAME. 2008;197(5):400–13.
 7.
Cortial J, Fahrat C, Guibas LJ, Rajashekhar M. Compressed sensing and timeparallel reducedorder modeling of structural health monitoring using DDDAS. Computational scienceICCS 2007. Berlin: Springer; 2007. p. 1171–9.
 8.
Filbet F, Negulescu C, Yang C. Numerical study of a nonlinear heat equation for plasma Physics. Int J Comp Math. 2012;89(8):1060–82.
 9.
Gander MJ, Hairer E. Nonlinear convergence analysis for the PARAREAL algorithm. In: Domain decomposition methods in Science and Engineering. 2008. vol. 60, p. 4556.
 10.
Gander MJ, Güttel S. PARAEXP: a parallel integrator for linear initialvalue problems. SIAM J Sci Comput. 2013;35(2):C123–42.
 11.
Gander MJ. 50 years of time parallel time integration. Multiple shooting and time domain decomposition. Berlin: Springer; 2015.
 12.
Hochbruck M, Lubich C. On Krylov subspace approximations to the matrix exponential operator. SIAM J Num Anal. 1997;34(5):1911–25.
 13.
Kalashnikova I, Barone MF. On the stability of a Galerkin reduced order model (ROM) of compressible flow with solid wall and farfield boundary treatment. IJNME. 2010;83:1345–75.
 14.
Kalashnikova I, Barone MF, Arunajatesan S, von Bloemen Waanders BG. Construction of energystable projectionbased reduced order models. Appl Math Comp. 2014;249:569–96.
 15.
Ladevèze P. Non linear computational structural mechanics new approaches and nonincremental methods of calculation. New York: SpringerVerlag; 1999.
 16.
Ladevèze P, Passieux JC, Néron D. The LATIN multiscale computational method and the proper generalized decomposition. Comput Methods Appl Mech Eng. 2010;199(21):1287–96.
 17.
Lions J, Maday Y, Turinici G. A “parareal” in time discretization of PDE’s. Comptes Rendus de l’Académie des Sciences, Séries I, Mathematics. 2001;332(7):661–8.
 18.
Maday Y, Nguyen NC, Patera AT, Pau GSH. A general multipurpose interpolation procedure: the magic points. Commun Pure Appl Anal. 2009;81:383–404.
 19.
Minion M. A hybrid parareal spectral deferred corrections method. Commun Appl Math Comput Sci. 2010;5(2):265–301.
 20.
Nievergelt J. Parallel methods for integrating ordinary differential equations. Comm ACM. 1964;7:731–3.
 21.
Prud’homme C, Rovas D, Veroy K, Maday Y, Patera AT, Turinici G. Reliable real time solution of parametrized partial differential equations: reducedbasis output bound methods. J Fluids Eng. 2002;124(1):70–80.
 22.
Willcox K, Peraire J. Balanced model reduction via the proper orthogonal decomposition. AIAA J. 2002;40(11):2323–30.
Author information
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Received
Accepted
Published
DOI
Keywords
 IMEX
 LATIN
 PARAEXP
 PARAREAL
 Performance
 Reduced order model