Recursive POD expansion for reactiondiffusion equation
 M. Azaïez^{1}Email author,
 F. Ben Belgacem^{2} and
 T. Chacón Rebollo^{3, 4}
https://doi.org/10.1186/s4032301600601
© Azaïez et al. 2016
Received: 24 November 2015
Accepted: 11 February 2016
Published: 9 March 2016
Abstract
This paper focuses on the lowdimensional representation of multivariate functions. We study a recursive POD representation, based upon the use of the power iterate algorithm to recursively expand the modes retained in the previous step. We obtain general error estimates for the truncated expansion, and prove that the recursive POD representation provides a quasioptimal approximation in \(L^2\) norm. We also prove an exponential rate of convergence, when applied to the solution of the reactiondiffusion partial differential equation. Some relevant numerical experiments show that the recursive POD is computationally more accurate than the Proper Generalized Decomposition for multivariate functions. We also recover the theoretical exponential convergence rate for the solution of the reactiondiffusion equation.
Keywords
Background
Model Reduction methods are nowadays basic numerical tools in the treatment of largescale parametric problems appearing in realworld problems. They are applied with success, for instance, in signal processing, analysis of random data, solution of parametric partial differential equations and control problems, among others. In signal processing, KarhunenLoève’s expansion (KLE) provides a reliable procedure for a low dimensional representation of spatiotemporal signals (see [12, 20]). Different research communities use different terminologies for the KLE. It is named the proper orthogonal decomposition (POD) in mechanical computation (see [3]), referred to as the principal components analysis (PCA) in statistics (see [17, 18, 24]) and data analysis or called singular value decomposition (SVD) in linear algebra (see [13]). These techniques allow large reduction of computational costs, thus making affordable the solution of many parametric problems of practical interest, otherwise out of reach. Let us mention some representative references [5, 6, 11, 16, 26, 27], although this list, by far, is not exhaustive.
The extension of KLE to the tensor representation of multivariate functions is, however, a challenging problem. Real problems are quite often multivariate. Let us mention, for instance the analysis of multivariate stochastic variables, simulation and control of thermal flows and multicomponent mechanics, among many others. Some recent techniques have been introduced to build lowdimensional tensor decompositions of multivariate functions and data. Among them, the HighOrder Singular Value Decomposition (HOSVD) provides lowdimensional approximation of tensor data, in a similar way as the Singular Value Decomposition allows to approximate bivariate data (see [7, 8, 21]). Also, the Proper Generalized Decomposition (PGD) appears to be well suited in many cases to approximate multivariate functions by lowdimensional varieties (see [14, 15]). However, in general there is not an optimal tensor of rank three or larger to approximate a given highdimensional tensor (see [9]).
In this paper we study an alternative method to build lowdimensional tensor decompositions of multivariate functions. This is a recursive POD (RPOD), based upon the successive application of the bivariate POD to each of the modes obtained in the previous step. In each step only one of the parameters is active, while the set of the remaining parameters is considered as a passive single parameter. We introduce a feasible version of the RPOD, in which the expansion is truncated whenever the singular values are smaller than a given threshold. This provides a fast algorithm, as only a small number of modes is computed, just those required to achieve a targeted error level.
As an application, we analyze the velocity of convergence of the RPOD applied to approximate the solution of the reactiondiffusion equation. We prove that the expansion converges with exponential rate. We use as main theoretical tool the CourantFischerWeyl Theorem, that allows to reduce the error analysis of the POD expansion to that of the polynomial approximation of the function to be expanded. We also use the analytic dependence of the solution on the diffusivity and reaction rate coefficients, that yields the exponential convergence rate. This analysis is based upon the one introduced in [2]. Further, we use subsequent bounds for the singular values to construct a practical truncation error estimator, which is used to recursively compute the expansion by the Power Iterate (PI) method [1]. This avoids to compute the full singular value decomposition of the correlation matrix, just the modes needed to attempt a given error threshold are computed. The PI method provides a fast and reliable tool to build the POD expansion of bivariate functions, of which we take advantage to recursively build the RPOD expansion.
We present a battery of numerical tests, in which we apply the RPOD to the representation of threevariate functions, and in particular to the solution of the reactiondiffusion. We compare the RPOD to the PGD expansion. The PGD expansion can be interpreted as the PI method applied to the effective computation of the POD for bivariate functions (see [23]). We here consider its extension to multivariate functions. We obtain exponential convergence rates for both RPOD and PGD expansions, although the RPOD is in general more accurate than the PGD for the same number of modes. We also recover an exponential rate of convergence for the RPOD approximation of the solution of the reactiondiffusion equation, in quite good agreement with the theoretical expectations.
The guidelines of the paper are as follows. “The KarhunenLoève decomposition on Hilbert spaces” section recalls the POD or KarhunenLoève expansion in Hilbert spaces. In “Recursive POD representation” section, we introduce the recursive POD decomposition of multivariate functions, and make a general error analysis. “Analysis of solutions of the reactiondiffusion” section deals with the error analysis for the RPOD expansion of the solution of the reactiondiffusion equation. Finally, in “Numerical tests” section we present some numerical tests where we analyze the practical performances of the recursive POD decomposition of multivariate functions.
Notation—Let \(X\subset {\mathbb R}^d\) be a given Lipschitz domain and G a measure space. We denote by \(L^2(G,X)\) the Bochner space of measurable and square integrable functions from G on X (cf. [10]).
The KarhunenLoève decomposition on Hilbert spaces
Consequently, there exists a complete orthonormal basis of H formed by eigenvectors \((v_m)_{m \ge 0}\) of A, associated to nonnegative eigenvalues \((\lambda _m)_{m \ge 0}\), that we assume to be ordered in decreasing value. Each nonzero eigenvalue has a finite multiplicity, and 0 is the only possible accumulation point of the spectrum. If H is infinitedimensional, then \(\lim _{m \rightarrow \infty } \lambda _m =0\).
Corollary 0.1
The main interest of the POD is the following bestapproximation property (cf. [22], Chapter 2):
Lemma 0.2
Recursive POD representation
Lemma 0.3
Proof
Feasible recursive POD representation
Algorithm FRPOD (Feasible recursive POD representation)

Step 1: Compute the modes \(\varphi _m\) and \(v_m\) and singular values \(\sigma _m\) for \(m=1,\ldots ,M_\varepsilon \), until \(\alpha _{M_\varepsilon } \le A \, \varepsilon \).

Step 2: For each \(m=1,\ldots , M_\varepsilon \), compute the modes \(u_m^{(k)}\) and \(w_k^{(m)}\) and the singular values \(\sigma _m^{(k)}\) for \(k=1,\ldots ,K_m\), until \(\beta _{K_m}^{(m)} \le B \, \varepsilon \).
Lemma 0.4
Proof
In practice we recursively compute the expansion by the PI method [1]. This avoids to compute the full singular value decomposition of the correlation matrix, we just compute the modes needed to reach a given error threshold.
Quasioptimality of recursive POD representation
Lemma 0.5
Proof
Note that in particular this implies that the POD expansion (15) is more accurate than the threevariate PGD one.
The following result states the quasioptimality of the feasible RPOD with representations.
Lemma 0.6
for any trivariate approximation \(\hat{T}_M\) of T with M modes, of the form (13).
Proof
Then, the feasible RPOD representation is more accurate than \(\hat{T}_M\), for \(\varepsilon \) small enough, if the inequality in (16) is strict. If (16) is an equality, this means that \(\hat{T}_M\) is optimal. In this case the accuracy of the feasible RPOD representation can be made arbitrarily close to the optimal one. It should be noted, however, that the RPOD contains more modes than \(\hat{T}_M\). Anyhow, we present some numerical experiments in “Numerical tests” section that show than the RPOD representation is more accurate than the PGD one, for the same number of modes.
Analysis of solutions of the reactiondiffusion
Our main result is the following.
Theorem 0.7
Therefore, the recursive POD expansion converges with spectral accuracy in terms of the number of truncation modes in the main and secondary expansions.
The proof of this result is essentially based upon the analyticity of T with respect to diffusivity \(\gamma \) and reaction rate \(\alpha \). It is rather technical, and will come up after several lemmas, the first of which is
Lemma 0.8
The mapping \((\gamma , \alpha ) \in \mathcal{G}\mapsto T_{(\gamma ,\alpha )}\in L^2(\mathcal{Q})\) is analytic.
Proof
According to (23), T is the sum of two contributions, coming from the initial condition \(T_0\) and the source f. We prove the analyticity for each of them.
Lemma 0.9
Remark 0.1
We now need to derive similar approximation estimates for analytic vector valued functions defined from \(\mathcal G\) into \(L^2(\mathcal Q)\). The following result holds
Lemma 0.10
Proof
Remark 0.2
Proof of Theorem 0.7
Remark 0.3

The constant \(C_\rho \) in estimate (24) also depends on the parameters domain \(\mathcal G\). We do not make explicit this dependence to simplify the notation.

The limit value for the convergence rates \(\rho _*\) only depends on the ratio \(\gamma _M/\gamma _m\), as$$\begin{aligned} \rho _*= \frac{2}{\sqrt{\frac{\gamma _M}{\gamma _m}} 1}+1. \end{aligned}$$

In view of estimate (24), in general a quasioptimal choice for I is \(I=M+\displaystyle \frac{1}{2}\log M\) (actually, the closest integer to this number). In this case,We thus obtain the same asymptotic convergence order when \(M \rightarrow \infty \) as for \(\Vert TT_M\Vert _{L^2(\mathcal{G}\times \mathcal{Q})} \).$$\begin{aligned} \Vert TT_P\Vert _{L^2(\mathcal{G}\times \mathcal{Q})} \le C_\rho \, \rho ^{M}. \end{aligned}$$

For more general parameterdepending parabolic equations, the above technique applies if the elliptic operator is symmetric. This allows to diagonalize the problem and expand the solution as a series in terms of the eigenfunctions of the elliptic operator. The use of Courant–Fischer–Weyl Theorem [19] allows to reduce the estimate of the truncation error of the POD expansion to the estimate of the interpolation error of the solution with respect to one of the parameters, eventually by polynomial functions. Then the convergence rate of the POD expansion will depend on the smoothness of the solution with respect to the parameters of the problem.
Reordering of recursive POD expansion
Practical implementation
Also, in view of (38) and (39), we deduce that a good estimator the error \(\Vert TT_P\Vert _{L^2(\mathcal{G}\times \mathcal{Q})}\) is \(\tau _I^{(M)}\), associated to the last computed mode, such that \(I+M=K\).
Numerical tests
This section is devoted to the comparison of the practical performances of the feasible RPOD expansion. In particular, we confirm the exponential rate of convergence of the truncated POD expansion for the diffusionreaction equation proved in section “Analysis of solutions of the reactiondiffusion”. We are also interested in comparing the rate of convergence of RPOD and PGD expansions, as the latter is particularly well suited to approximate multivariate functions. We have considered functions with high and low smoothness, as the smoothness plays a crucial role in the decreasing of the size of the modes in both expansions. In addition we have tested the ability of both representations to approximate functions that already have a separated tensor structure. For completeness we describe in the Appendix the application of the PGD expansion to approximate multivariate functions.
Multivariate functions
In this test we apply the RPOD and the PGD to approximate multivariate functions. Actually we consider trivariate functions a generic test to determine the relative performances of both expansions. We have considered the following tests:
The space domain is fixed to \(\Omega =X \times Y \times Z\), with \(X=Y=Z=]1,1[\) and Gauss–Lobatto–Legendre quadrature is used (see [4]) with the polynomial degree equal to \(N=64\). These formulas are used to evaluate the matrix representation of the operators B and A.
We set the tolerance error in \(L^2(X \times Y \times Z)\) in the residual of both RPOD and PGD expansions to \(\mu = 10^{7}\). This corresponds to \(\varepsilon =10^{14}\) in Algorithm FRRPOD. We have displayed in Figs. 1, 2, 3 the comparison of the convergence history of the feasible RPOD and PGD processes, for all the threevariate functions considered. The xaxis represents the number of eigenmodes while the yaxis represents the \(L^2(X \times Y \times Z)\) error, in logarithmic coordinates. We observe in Fig. 1 that the RPOD just needs 3 modes to fit a function that already has a separated tensor structure, while the PGD requires 17 modes to reach the error level. Further, that for functions with low smoothness both expansions require approximately the same number of modes to reach a moderate accuracy, however the RPOD is more efficient to reach high accuracy in all cases. Finally, that the error associated to the RPOD expansions is almost in all cases below the one associated to the PGD one for the same number of modes.
Comparison of feasible RPOD and PGD for trivariate functions
Modes RPOD  Modes PGD  

Case 1  3  17 
Case 2  9  17 
Case 3  21  43 
ReactionDiffusion equation
This part is devoted to determining the effective convergence rate of the RPOD approximation of some solutions to the transient reactiondiffusion equation when parameterized by the diffusivity and reaction coefficients. We assess the exponential convergence rate and investigate the variation of this rate with respect to the set \(\mathcal{G}=[\gamma _m,\gamma _M]\times [\alpha _m,\alpha _M]\).
Test 1: Exponential convergence rate.
Figure 4 shows the convergence history of the RPOD expansion for the reactiondiffusion equation (40) in terms of the total number of modes in the expansion. We have considered the sets of diffusivities \(\gamma \in [1,51]\), and reaction rates \(\alpha \in [0,100]\). The error is measured in \(L^2(\mathcal{Q})\) norm. The numbers of secondary modes \(I_m\) has been determined to fit the test \(\sigma _m^{(I_m+1)} \le \varepsilon =10^{10}\). In practice a small amount of secondary modes (actually, \(I_m \simeq 4\)) is needed to fit this test. The modes have been rearranged in decreasing order of the effective singular values \(\tau _i^{(m)}=\sigma _m \, \sigma _i^{(m)}\) (denoted by a \(\CIRCLE \) symbol). We observe that the \(\tau _i^{(m)}\) indeed are good error estimators for this rearranged expansion, as was argued in “Practical implementation” section.
Test 2: Dependence of the convergence rate with respect to the parameters range.
Computed and theoretical convergence rates, for different values of \(R=\gamma _M/\gamma _m\) and fixed \(\alpha _m=0\), \(\alpha _M=100\) (for Data 3)
\(R=\gamma _M/\gamma _m\)  \(\alpha _c\)  \(\alpha _*\) 

25  1.48  0.81 
36  1.44  0.67 
64  1.38  0.50 
100  1.36  0.40 
400  1.35  0.20 
Conclusion
We have introduced in this paper a recursive POD (RPOD) expansion to approximate multivariate functions. The approach consists in building truncated recursive POD expansions of the modes that appear in the expansions at the previous level, to a given tolerance error. We have constructed a practical truncation error estimator by means of bounds for the singular values, which is used to recursively compute the expansion by the Power Iterate (PI) method. This allows to compute just the modes needed to attempt a given error threshold. We have proved the quasioptimality of this RPOD expansion in \(L^2\), similar to that of the POD expansion.
We have proved the exponential rate of convergence of the RPOD expansion for the solution of the reactiondiffusion equation, based upon the analyticity of its solution with respect to those parameters.
We have finally performed some relevant numerical tests that on one hand show that the RPOD is more accurate than the PGD expansion for threevariate functions, and that on another hand confirm the exponential rate of convergence for the solution of the reactiondiffusion equation, presenting a good agreement with the qualitative and quantitative theoretical expectations.
Further extensive tests for more complex multivariate functions, in particular of practical interest for engineering applications, are in progress and will appear in a forthcoming paper.
Declarations
Authors' contributions
MA, FBB and TCR participated to the development of the mathematical proves and the numerical investigations. They checked the results and wrote the manuscript. All authors read and approved the final manuscript.
Acknowledgements
None.
Competing interests
The authors declare that they have no competing interests.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Authors’ Affiliations
References
 Azaïez M, Belgacem Ben F. KarhunenLoève’s truncation error for bivariate functions. Comput Methods Appl Mech Eng. 2015;290:57–72.View ArticleGoogle Scholar
 Azaiez M, Ben Belgacem F and Chacón Rebollo T. Error bounds for POD expansions of parameterized transient temperatures. Submitted to Comp. Methods App. Mech. Eng.Google Scholar
 Berkoz G, Holmes P, Lumley JL. The proper orthogonal decomposition in the analysis of turbulent flows. Annu Rev Fluids Mech. 1993;25:539–75.MathSciNetView ArticleGoogle Scholar
 Bernardi C, Maday Y. Approximations spectrales de problèmes aux limites elliptiques, Mathématiques et applications. Berlin: Springer; 1992.Google Scholar
 Chinesta F, Keunings R, Leygue A. The Proper Generalized Decomposition for Advanced Numerical Simulations: A Primer. New York: Springer Publishing Company, Incorporated; 2013.MATHGoogle Scholar
 Chinesta F, Ladevèse P, Cueto E. A Short Review on Model Order Reduction Based on Proper Generalized Decomposition. Arch Comput Methods Eng. 2011;18:395–404.View ArticleGoogle Scholar
 De Lathauwer L, De Moor B, Vandewalle J. A multilinear singular value decomposition. SIAM J Matr Anal Appl. 2000;21(4):12531278.MathSciNetMATHGoogle Scholar
 De Lathauwer L, De Moor B, Vandewalle J. On the Best RankOne and RankR1; R2;.; RN Approximation of Higher Order Tensors. SIAM J Matr Anal Appl. 2000;21(4):13241342.Google Scholar
 De Silva V, Lim LH. Tensor rank and the ill posedness of the best lowrank approximation problem. SIAM J Matrix Anal Appl. 2008;20(3):1084–127.MathSciNetView ArticleMATHGoogle Scholar
 Diestel J and Uhl. Vector measures. AMS J J; 1977.Google Scholar
 Epureanu BI, Tang LS, Paidoussis MP. Coherent structures and their influence on the dynamics of aeroelastic panels. Int J NonLinear Mech. 2004;39:977–91.View ArticleMATHGoogle Scholar
 Ghanem R and Spanos P. Stochastic finite elements: a spectral approach. SpringerVerlag; 1991.Google Scholar
 Golub GH, Van Loan CF. Matrix Computations. 3rd ed. Baltimore: The Johns Hopkins University Press; 1996.MATHGoogle Scholar
 Heyberger C, Boucard PA, Néron D. A rational strategy for the resolution of parametrized problems in the PGD framework. Comp Meth Appl Mech Eng. 2013;259:40–9.MathSciNetView ArticleMATHGoogle Scholar
 Heyberger C, Boucard PA, Néron D. Multiparametric Analysis within the Proper Generalized Decomposition Framework. Comput Mech. 2012;49(3):277–89.View ArticleMATHGoogle Scholar
 Holmes P, Lumley JL, Berkooz G. Coherent Structures, Synamical Systems and Symmetry, Cambridge Monographs on Mechanis. Cambridge: Cambridge University Press; 1996.Google Scholar
 Hotelling H. Analysis of a complex of statistical variables into principal componentse. J Educ Psychol. 1933;24:417–41.View ArticleGoogle Scholar
 Jolliffe IT. Principal Component Analysis. Springer; 1986.Google Scholar
 Little G, Reade JB. Eigenvalues of analytic kernels. SIAM J Math Anal. 1984;15:133–6.MathSciNetView ArticleMATHGoogle Scholar
 Loève MM. Probability Theory. Princeton: Van Nostrand; 1988.MATHGoogle Scholar
 Lorente LS, Vega JM, Velazquez A. Generation of Aerodynamic Databases Using HighOrder Singular Value Decomposition. J Aircraft. 2008;45(5):1779–88.View ArticleGoogle Scholar
 Muller M. On the POD Method. An Abstract Investigation with Applications to ReducedOrder Modeling and Suboptimal Control, Ph D Thesis. GeorgAugust Universität, Göttingen; 2008.Google Scholar
 Nouy A. A generalized spectral decomposition technique to solve a class of linear stochastic partial differential equations. Comput Meth Appl Mech Eng. 2007;196:4521–37.MathSciNetView ArticleMATHGoogle Scholar
 Pearson K. On lines and planes of closest fit system of points in space. Philo Mag J Sci. 1901;2:559–72.View ArticleMATHGoogle Scholar
 Trefethen LN. Approximation theory and approximation practice. Software, Environments, and Tools. Philadelphia: Society for Industrial and Applied Mathematics (SIAM); 2013.MATHGoogle Scholar
 Willcox K, Peraire J. Balanced Model Reduction via the Proper Orthogonal Decomposition. AIAA. 2002;40:2323–30.View ArticleGoogle Scholar
 Yano M. A SpaceTime PetrovGalerkin Certified Reduced Basis Method: Application to the Boussinesq Equations. SIAM J Sci Comput. 2013;36:232–66.MathSciNetView ArticleGoogle Scholar