Skip to main content

Physics-Informed Neural Network water surface predictability for 1D steady-state open channel cases with different flow types and complex bed profile shapes

Abstract

The behavior of many physical systems is described by means of differential equations. These equations are usually derived from balance principles and certain modelling assumptions. For realistic situations, the solution of the associated initial boundary value problems requires the use of some discretization technique, such as finite differences or finite volumes. This research tackles the numerical solution of a 1D differential equation to predict water surface profiles in a river, as well as to estimate the so-called roughness parameter. A very important concern when solving this differential equation is the ability of the numerical model to capture different flow regimes, given that hydraulic jumps are likely to be observed. To approximate the solution, Physics-Informed Neural Networks (PINN) are used. Benchmark cases with different bed profile shapes, which induce different flows types (supercritical, subcritical, and mixed) are tested first. Then a real mountain river morphology, the so-called Step-pool, is studied. PINN models were implemented in Tensor Flow using two neural networks. Different numbers of layers and neurons per hidden layer, as well as different activation functions (AF), were tried. The best performing model for each AF (according to the loss function) was compared with the solution of a standard finite difference discretization of the steady-state 1D model (HEC-RAS model). PINN models show good predictability of water surface profiles for slowly varying flow cases. For a rapid varying flow, the location and length of the hydraulic jump is captured, but it is not identical to the HEC-RAS model. The predictability of the tumbling flow in the Step-pool was good. In addition, the solution of the estimation of the roughness parameter (which is an inverse problem) using PINN shows the potential of this methodology to calibrate this parameter with limited cross-sectional data. PINN has shown potential for its application in open channel studies with complex bed profiles and different flow types, having in mind, however, that emphasis must be given to architecture selection.

Introduction

Besides its extensive use for classification problems and for the search of patterns in data, Machine Learning techniques (ML) have shown a great capability as surrogate models to approximate the behavior of both artificial and natural systems. ML can find non-linear complex spatio-temporal functional relations for the big-data regimes [1, 2]. Nevertheless, ML has certain drawbacks affecting its performance. Firstly, it does not consider the system physics. Secondly, it depends on the quantity and quality of data to be robust and to attain convergence [1, 3]. In fact, in natural systems, the available data may be scarce because of the difficulty of measuring. To address this challenge, machine learning techniques can take advantage of the knowledge embedded in the laws of physics [2]. This notion leads to the approach known as Physics-Informed Machine Learning. In particular, Physics-Informed Neural Networks (PINN) have been applied to solve both forward and inverse problems. Forward problems deal with the solution of Initial Boundary Value Problems [3, 4]. An inverse problem tackles the inference of quantities of interest such as parameters or hidden states of a system using a limited and potentially noisy set of observed data [3,4,5].

An interesting feature of PINN is that the evaluation of derivatives is performed through automatic differentiation (AD) [6]. AD consists of a family of techniques in which the evaluation of derivatives is exact without resorting to symbolic differentiation [7]. In addition, PINN does not require discretization points. In that sense, it can be classified as a meshless method. Collocation points, where the differential equations are evaluated, need to be provided [3].

Physics-informed machine learning has been used in many studies related to hydrodynamics [8, 9]. Mao et al. [4] solved 1-D and 2-D Euler equations for high-speed aerodynamic flow with Physics-Informed Neural Network (PINN). The results were not superior to traditional techniques for forward problems, but PINN results were superior in inverse problems. Guo et al. [10] tested PIDL prediction capacity to solve different partial differential equations (PDE): 1-D wave equation, kdV Burger's equation, and Two-soliton solution of the korteweg-De Vries Equation. In all cases, PIDL provides good predictability. However, the authors have not been able to find any application of PINN for mountain rivers. The modeling of a mountain river reach is a challenging task [11]. A mountain river model must be able to deal simultaneously with Gradually Varied Flows (GVF, either only subcritical or only supercritical) and Rapid Varying Flows (RVF, transcritical: both subcritical and subcritical regimes are observed) [12]. GVF present a slow variation of the flow depth profile with parallel streamlines. RVF have a fast change of water depth with streamlines having a pronounced curvature producing discontinuities in the solution (hydraulic jumps). RVF can produce spurious oscillations around discontinuities in a numerical model [11, 13]. To explore the ability of PINN to deal with these problems, different open channel cases with increasing complexity have been tested in this study.

The first two benchmark cases present GVF, where the solution is smooth. Thus, these cases were helpful to ensure that the developed PINN method provide correct answers. The next two benchmark cases deal with RVF. These cases were used to test the solution stability in case of discontinuities, which is crucial for the real cases. For RVF, two transitions were tested: supercritical to subcritical and subcritical to supercritical. All the previous cases give a clear picture of the PINN predictability performance. Then, it was tested in a more complex application: A mountain river reach. The natural system under analysis was a morphology called Step-pool, which is frequently found in mountain streams when bed slope varies from 0,04–0,2 [14]. Step-pools are an alternation of step-pool units having a stair-case shape [14]. A step-pool unit has a step commonly formed by boulders and cobbles but other materials such as large wood debris or bedrock are also found [15], and a pool having finer material [14]. This morphology regulates flow resistance through a tumbling flow [15]. A tumbling water flow, over or through steps, is supercritical until it falls into a pool and changes to a subcritical flow after a hydraulic jump [16]. Below steps is the place with the higher turbulence producing energy dissipation due to roller eddies, hydraulic jumps and velocity fluctuations [15, 16].

In this study, the steady case is considered, so the Energy Differential Equation (EDG) is used. The same equation is solved for this case by a widely used hydrodynamic model HEC-RAS [17]. EDG is expected to work well for GVF. However; EDG is not valid in RVF [17], so spurious oscillations are expected. Indeed, the well-known software HECRAS uses a steady version of the momentum equation under some RVF conditions. 

The remaining of this article is organized as follows. "Materials and methods" section provides the materials and methods applied in this article, including a description of the five studied cases, the PINN architectures implemented, a description of the HEC-RAS model and the metrics to compare PINN and HEC-RAS results. "Results” section compares the results of different PINN architectures, taking the HEC-RAS model as a baseline. An analysis of the performance of activation functions, neural network dimension, and PINN predictability is done in the "Discussion” section. "Conclusions" section highlights the main findings in the current research.

Materials and methods

Cases under study

Benchmark cases

PINN predictability was tested for four benchmark open channel cases with prismatic cross sections for the forward problem. Each case has different bed shapes producing different water surface profiles. The main idea is to test the ability of the method to approximate the solution of the differential equation for different flow regimes. Case 1 is intended to represent the longitudinal profile of a river with a changing bed slope [18]. Figure 1a depicts the bed profile having different inflection points. The cross-section is rectangular with a width (B) of 10 m, a Manning’s roughness value (n) of 0.03, and a flow (Q) of 15 m3 s−1. In this case there is GVF in the whole channel; moreover, the flow regime is known to be subcritical.

Fig. 1
figure 1

Bed profile for different cases.

Case 2 represents a case of rapid flow in a spillway where the slope increases downstream (Refer Fig. 1b)[18]. The flow regime in this case is also GVF, but now it is supercritical. The cross-section and Manning’s roughness values are the same as in Case 1, but the flow is 22 m3 s−1.

Figure 1c shows the profile for Case 3, which consists of two parts separated by an inflection point. The first part is 200 m long with a slope of 0.025. The second part is 600 m long, having a slope of 0.0002. The sudden change in bed slope results in the presence of a hydraulic jump (a transition from supercritical to subcritical flow), i.e., a RVF. The cross-section in this reach is trapezoidal with a base of 2.5 m, a lateral slope of 0.8, n is 0.012, and the flow is 25 m3 s−1.

Figure 1d depicts Case 4. It is composed of two reaches. The first reach is 1000 m long with a slope of 0.0006, and the second one is 200 m long with a slope of 0.015. The sudden slope increase produces the flow depth profile to suddenly decrease, passing from subcritical to supercritical flow, a RVF as in the previous case. As in Case 3, Case 4 has a trapezoidal cross-section with a width of 1 m, a lateral slope of 1, an n value of 0.018 and a discharge of 6 m3 s−1.

Real case

Figure 1e depicts the profile of the step-pool under study having two step-pool units. The chosen morphology to be studied is Step-pool 1 in Fig. 2. This is part of a hydraulic observatory where different morphologies are studied in the headwaters of the mountain Quinuas river, in Southern Ecuador. This reach is 12.22 m long and has a mean slope of 6.1%. Moreover, this Step-pool has been used in previous studies by the authors [19, 20]. The available data consist of mean velocity and water depth for different flow magnitudes so that different flow resistance conditions can be studied. Moreover, topography and bed composition are available as well. This kind of morphology has been selected for its flow characteristics, as mentioned in the Introduction, given that its prediction poses a challenge to any numerical model.

Fig. 2
figure 2

Plan view of the studied 1500 km river reach showing the sequence of the sub-reaches and the location of the meteorological station

Three flow magnitudes have been chosen for the current research based on the data available in Cedillo et al. [19]: 0.035 m3s−1, 0.443 m3s−1, and 0.878 m3s−1 with the respective effective roughness coefficients: 0.414, 0.193, and 0.134.

Physical informed neural network (PINN)

The scheme shown in Fig. 3 schematizes the PINN architecture used for our study. It is trained using a loss function that includes both data and the evaluation of the governing differential equation at collocation points. This combination has demonstrated to be exceptionally well suited for the solution of physical equations governing a given phenomenon, as well as for the corresponding inverse problem. The Energy Differential Equation (EDE) (Eq. (1)) was used as the main information for PINN since all the cases are run under steady-state conditions [18].

Fig. 3
figure 3

Physics-Informed Deep Learning sheme

It is important to mention that the governing equation was written using centimeters as length units for the water depth and the bed level. This was necessary because during exploratory tests the resulting water levels of PINN did not approximate correctly the analytical solutions (AS). Take into account that the way the PINN method enforces the different physical principles and constraints is by means of a loss function that has to be optimized. Thus, it is necessary for convergence that the terms of this loss function are of the same order of magnitude. In the above-mentioned exploratory studies, it was observed that gradients in the differential equation had a lower order of magnitude than other terms of the loss function. After the above-mentioned modification of units, PINN started converging to good solutions.

The Energy Differential Equation (EDE) has the following form:

$$\frac{{dz_{o} }}{{dx_{o} }} + \frac{dh}{{dx}} + \frac{{d\left( {\frac{{Q^{2} }}{{2 g A^{2} }}} \right)}}{dx}*100 + \frac{{Q^{2} n^{2} }}{{R^{\frac{4}{3}} A^{2} }}*100 = 0 = f$$
(1)

where zo is the bed level (cm), xo is the distance (m), h is the water depth (cm), Q is the flow m3s−1, R hydraulic radius (m), and A is the cross-sectional area (m2). The first term in Eq. (1) represents the change in bed slope elevation (zo) with the distance (xo). The next term is the water depth slope, followed by the change in kinetic energy. The last term deals with the friction loss.

The PINN models were implemented in TensorFlow ® [21]. The code is based on the one shared by Raissi et al. [3]. Following the procedure of Kissas et. al [22], two neural networks sharing hyper parameters were used. The first one deals with the complex geometry present in each case, the second one is for the prediction of water depth. Each described case was run with different PINN models varying the number of hidden layers (3, 5, 7, 9 and 11), the number of neurons per hidden layer (10, 20, 40 and 60), and the activation functions: Hyperbolic Tangent (Tanh), ReLU, Sigmoid, and Sin. Tanh is a zero-centered AF whose output varies between − 1 and 1 [23]. According to Nwankpa et al. [24], this aids in backpropagation. Tanh suffers saturation when the input tends to ± ∞, resulting in a vanishing gradient where the weights are not updated during backpropagation [23]. ReLU output is always positive [25], producing bias in the next layer [23]. Moreover, it is left-hand saturated, and only a certain number of neurons are active [24]. Sigmoid values range between 0 to 1 [25], which has the same bias problem as ReLU. The Sin AF has been selected based on [26], who advise not to limit the considered AF`s to popular ones.

Forward problem: solving the differential equation

In the forward problem, the main information for the loss function comes from the evaluation of the governing equation at collocation points. Water depth data is provided only at the boundary conditions (BC). The loss function is computed using the mean square error (MSE) metric, including the data at BC:

$${\text{Loss function }} = {\text{MSE}}_{{{\text{BC}}}} + {\text{ MSE}}_{{\text{f}}} + {\text{ MSE}}_{{\text{G}}}$$
(2)

where

$${\text{MSE}}_{{{\text{BC}}}} {\text{ = 1/N}}_{{{\text{BC}}}} { }\mathop \sum \limits_{{\text{i = 1}}}^{{{\text{N}}_{{{\text{BC}}}} }} \left| {{\text{h(x}}_{{{\text{BC}}}}^{{\text{i}}} {\text{) - h}}^{{\text{i}}} {)}} \right|$$
(3)

\(h\left( {x_{BC}^{i} } \right)\) denotes the training data (water depths) at the boundaries, hi are the predictions of PINN value at BC, and NBC the number of training data;

$${\text{MSE}}_{{\text{f}}} {\text{ = 1/N}}_{{\text{f}}} { }\mathop \sum \limits_{{\text{i = 1}}}^{{{\text{N}}_{{\text{f}}} }} \left| {{\text{f(x}}_{{\text{f}}}^{{\text{i}}} {)}} \right|$$
(4)

xfi are the collocation points where the Differential Equation is evaluated and Nf is the number of collocation points;

$${\text{MSE}}_{{\text{G}}} {\text{ = 1/N}}_{{\text{G}}} \mathop \sum \limits_{{\text{i = 1}}}^{{{\text{N}}_{{\text{G}}} }} \left| {{\text{z}}\left( {{\text{x}}_{{\text{G}}}^{{\text{i}}} } \right){\text{ - z}}^{{\text{i}}} } \right|$$
(5)

xGi are points where the bed elevations are available, zi are the predictions of geometric points of PINN, and NG is the number of geometric points available.

Step-pool brings an additional difficulty besides a complex profile: the cross-sections are not prismatic. Our proposed solution is to obtain an equivalent cross-section and then adjust cross sectional area (A) and hydraulic radius (R) data to an exponential equation which are used for discharge-stage relations. The computation of the equivalent cross-sections requires the following steps. First, cross-sections are measured at the studied reach; second, the cross-sections coordinates are translated so that the deepest point is located at the origin; third, each elevation of the equivalent cross-section is the geometric mean of the corresponding points of the measured cross-sections. Geometric mean is not sensible towards outliers [27, 28] being useful in highly varied cross sections in natural rivers.

Inverse problem

The inverse problem was only solved for Case 5: The Step-pool. In this case, instead of having water depths at BC, there are water level measured at three points inside the domain. Those values are used instead BC values in Eq. (3) so that this loss function component enforces measured field data. In the inverse problem, the water level profile as well as the roughness factor are found. The roughness values are compared with the effective roughness coefficients found in Cedillo et al. [19] with GLUE methodology. The scope of inverse problem is to analyze the predictive capacity of the roughness parameter under different resistance conditions and with limited cross-sectional data.

HEC RAS model

HEC-RAS® is a hydrodynamic model widely used in different studies [29,30,31,32]. Furthermore, these model results have been used as the benchmark for PINN results. All the models have been run under steady-state conditions. Case 1 was run under subcritical flow, Case 2 was run with supercritical flow, and Case 3, Case 4, and Case 5 were run with a mixed flow regime. Under these conditions, HEC-RAS solves the energy equation between two consecutive cross-sections (Eq. (6)) [17]:

$${\text{z}}_{{2}} \, + \,{\text{h}}_{{2}} \, + \,\alpha_{{2}} {\text{U}}_{{2}}^{{2}} /{2} {\text{g}}\, = \,{\text{z}}_{{1}} \, + \,{\text{h}}_{{1}} \, + \,\alpha_{{1}} {\text{U}}_{{1}}^{{2}} /{ 2} {\text{g}}\, + \,{\text{h}}_{{\text{e}}}$$
(6)

z1, z2 are bed levels; h1, h2 are water depth; U1, U2 are velocities; α1, α2 are velocity weighting coefficients; and he is the energy head loss. The parameter he has, in principle, two components: expansion or contraction losses and friction losses. All the studied cases have prismatic XS, so there are no expansion–contraction losses.

Direct step method

In Case 3 an additional solution method called “Direct Step Method” is used [33]. This method consists of the solution of The Energy Differential Equation (EDE) (Eq. (1)) by using finite differences. This methodology is applied in this case because of the discontinuity (hydraulic jump) location procedure used. The hydraulic jump location is determined through an iterative process where the initial depth upstream (yi) and subsequent depth downstream (ys) must coincide with the values of equation given by Marriot et al. [33] which relate both values. This entails using and ad hoc strategy once hydraulic jumps are detected. For PINN, we do not use any ad hoc procedure.

Metrics

Three metrics are used to compare the PINN predictions and the HEC-RAS model results. Each metric analyzes different aspects of the difference between both models (residuals). First, Root Mean Square Error (RMSE) is an average of the residuals between PINN and HEC-RAS model, giving more weight to higher residuals [34, 35]. Second, MAE is an average of the residuals, where all the residuals have the same weight [34]. Third, the Nash–Sutcliffe efficiency index (EF) is a reliable and flexible metric used as an indicator of fitness goodness [36,37,38]. Moreover, Ritter & Muñoz-Carpena [39] provides a table to interpret the fitting quality based on the EF value. Both RMSE and MAE were divided by the mean of the observations and multiplied by 100 to have dimensionless metrics.

Systematic studies

In order to gain a deeper understanding of PINN as a numerical method for the solution of the differential equation treated in this study, we have performed several numerical studies in a systematic way to analyze both the rate of convergence and the robustness of the approach. For the latter, we have performed a sensitivity analysis.

Rate of convergence

The convergence rate in a numerical method can be determined by finding the relation between the log of the error norm of the solution (L2-norm [40]) and the discretization size (related to the number of grid points). The slope of that relation is called rate of convergence. The rate of convergence indicates the rate at which the error decreases as the number of grid points increases [41].

As stated before, PINN does not have grid points, sensu stricto. Instead, it has the so-called collocation points, where the governing equations are imposed through the loss functions. Hence, we have computed different approximations of the solution with different number of collocation points. The L2-norm of the solution error was found for each run, allowing for the determination of rates of convergence for the different studied cases. This study was performed for each case with the activation function rendering the lowest loss function at convergence.

Sensitivity test

A sensitivity test has been performed by introducing a “perturbation” at the boundary conditions. That “perturbation” consist on a certain modification of the water depth values at the boundary condition: ± 2%, ± 4%, ± 6%, ± 8%, and ± 10%. The idea is to see the effect of this perturbation on the solution.

Results

Equivalent cross-section

Figure 4 shows three different measured cross-sections of the Step-pool. In addition, that Figure also shows that the equivalent cross-section tends to follow a central tendency, where the outliers do not play an important role. Furthermore, Fig. 5a, b shows both the Area and Hydraulic Radius of the equivalent cross-section fit well to an exponential equation having R2 values higher than 0.9.

Fig. 4
figure 4

Cross sections in Step-pool and equivalent cross section

Fig. 5
figure 5

Area and Hydraulic Radius data of the equivalent cross-section. a Area, b Hydraulic Radius

Forward PINN: solving the differential equation

We analyze now the performance of PINN as a discretization method for a Differential Equation that models the behavior of a stationary 1D fluid in an open channel. The idea is to study several aspects of the approximation space subjacent to the PINN method, which is directly related to the architecture of the Neural Networks used. We start by analyzing the performance of several activation functions.

Activation function performance

Several PINN models were run with different combinations of number of layers (depth), neurons per layer (width), and activation functions (AF). We consider the best model for each AF as the one that has reached the lowest value for loss function during the training process.

The ReLU activation function is the one with the lowest predictive performance in all the cases under study, having the highest values of the RMSE and MAE statistics in cases where hydraulic discontinuities are present—RVF flow (refer to Table 1). Moreover, EF depicts that most of the ReLU models have a “Unsatifactory” predictability except for Case 2 where the goodness-of-fit is “Very good” [39]. Case 2 has the smoothest solution (GVF) with a constant descending pattern downstream (Fig. 6b). The remaining cases have peaks and minima in the solution. Furthermore; Figs. 6, 7 display ReLU predicting a completely different response pattern than Sigmoid, Sin, Tanh, or HEC-RAS for benchmark as well as for the real case.

Table 1 Best cases of activation functions based on loss function
Fig. 6
figure 6

PINN results with the best case for each activation function for: a Case 1, b Case 2, c Case3, and d Case 4

Fig. 7
figure 7

PINN results with the best case for each activation function for Step-pool: a Low flow, b Mid flow, and c High Flow

Looking at Table 1, the prediction quality of Sigmoid, Sin, and Tanh is almost the same for all the cases according to RMSE and MAE. Moreover, EF shows a “Very Good” goodness-of-fit for most studied Cases. However, in Case 3, these AFs provide “Unsatisfactory” predictions. The “Direct Step Method” provides the best fitting according to Table 1, with lower RMSE and MAE values than PINN results and an EF value of 0.796 rendering a fitting performance deemed as “Acceptable”. Figure 6c and d shows the presence of oscillations near discontinuities in benchmark cases with RVF when Sigmoid and Sin are used. Sigmoid, Sin, and Tanh have produced promising results predicting almost the same water depth profile in the real case (Fig. 7a–c).

There are some predictability aspects to consider while using PINN. In Case 3 (Fig. 6c), three of the activation functions, Tanh, Sin, and Sigmoid, were able to predict the presence of the discontinuity (hydraulic jump) downstream from the place where the HEC-RAS model predicted it. None of them was able to accurately locate the position of the discontinuity on the second part of the reach. The “Direct Step Method” was not able to predict the position of the hydraulic jump either, but it was the closest to the HEC-RAS result. In the real case (Case 5, Fig. 7), the performance of PINN improves as flow increases according to RMSE and MAE values. Furthermore, the predictability of PINN in this real case is very good according to EF values [39] except for ReLU AF. However, Fig. 7 presents discrepancies between the results of PINN and HEC-RAS which is not the case in benchmark cases.

Based on the above analysis, Tanh has been the activation function with the highest resilience for all the studied cases when an important number of collocation points are used. Thus, the analysis of the neural network's size will be based on the models using Tanh. On the other hand, ReLU has been the activation function with the worst performance providing acceptable water depth predictions only in Case 2.

Neural network architecture

According to the previous section, the neural network architecture analysis is based on the best performing models, i.e., those using Tanh as AF. The optimal combination of hidden layers and neurons per hidden layer varies with the case under study. According to Table 1, a similar number of hidden layers are used in Cases 1, 2, and 3 having a variation of 5 and 7. However, the number of neurons per hidden layer ranges from 20 to 60. Case 4 has the lowest number of layers and neurons: 3 hidden layers with 20 neurons per hidden layer. Case 5 has 40 neurons per hidden layers being inside the range of Cases 1, 2, and 3, but the number of hidden layers is 3 which is lower than the previous mentioned cases. Moreover, it is important to notice that the remaining AFs in Case 5 are inside the number of hidden layers and neurons per hidden layer given by Cases 1, 2, and 3.

Inverse PINN

Figure 8 and Table 2 provides a comparison of the roughness values found by PINN with different AFs and the ones obtained through GLUE experiments (effective roughness) in Cedillo et al. [19]. Figure 8 displays that PINN roughness values follow the descending effective roughness pattern as flow increases. For low flow, Table 2 shows that Tanh, Sin, and Sigmoid provide similar roughness values, but lower than GLUE results. Furthermore, ReLU gives a completely different roughness value, lower than the rest of models. For mid and high flow, all the AFs seem to provide the same roughness values close to the GLUE ones. Moreover, Table 2 depicts that ReLU results are not trustful because of the high loss function value being one thousand times higher than the rest of AFs.

Fig. 8
figure 8

PINN results with the best case for each activation function for Step-pool, high flow

Table 2 Inverse PINN comparison between calibrated roughness values from GLUE experiments and the ones obtained from different AFs and different flow values

Looking at Table 2, it is apparent that the hidden layers (ranging from 3 to 9) with the lowest loss function covers a wide range of the tested cases; however the number of neurons per hidden layer is rather limited to high values such as 40 and 60. Tanh, Sin and Sigmoid provides almost the same values for low and high flow, but at mid flow Tanh got a value which is inside the roughness value of GLUE experiment.

Results of the systematic studies

Rate of convergence

Figure 9a–c provides the plot of L2-norm against the number of collocation points used in PINN. The slope of each plot provides the rate of convergence. Cases with smooth solution have different rate of convergence depending on the flow regime. Case 2 (supercritical flow: shallow and rapid flow; see Fig. 9b), has the highest rate of convergence: 1.4. On the other hand, Case 1(subcritical flow: deep and slow flow; see Fig. 9a) shows a rate of convergence of 0.4. Cases with discontinuities present different rates of convergence depending on the type of discontinuity. Case 3 (Fig. 9c) and Case 5 (Fig. 9e) discontinuity consist of a sudden water depth increase (hydraulic jump) having a rate of convergence of 0.2. In contrast, in Case 4 (Fig. 9d) the discontinuity has a sudden decrease of water depth. This Case has a higher rate of convergence: 0.7.

Fig. 9
figure 9

Rate of convergence. a Case 1, b Case 2, c Case 3, d Case 4, and e Case 5

Sensitivity test

Figure 10a–e presents the L2-norm when a “perturbation” of ± 10% is introduced in the water depth at the boundary condition. The results depict that the addition of the perturbation has effects on the model fitting performance, but, more importantly, a different behavior is observed depending on the flow regime. In Case 4 (sudden water depth decrease), and Case 5 (Step-pool), there are performance oscillations when the boundary water depths are increased or decreased. In Case 1 (Subcritical flow), the performance stays constant until reaching 6% of water depth increase. When the boundary water depth increases more than 6%, the performance decreases rapidly. However, there are performance oscillations when the boundary water depth decreases. In Case 2 the decrease of fitting performance when BC is decreased follows a pattern that seems parabolic, but this pattern is close to a line when the perturbation is positive. Case 3 (sudden increase of water depth) shows no sensitivity for negative variations in the water depth at the boundary; however a positive variation yields the highest performance variations.

Fig. 10
figure 10

Sensibility test. a Case 1, b Case 2, c Case 3, d Case 4, and e Case 5

Discussion

Activation function performance

The selection of activation functions (AF) is important for the predictability of a neural network [24, 42], so its analysis is an important issue [25]. An AF introduces non-linearity into the network [24]. Hence, neural networks can learn complex relationships between input and output [43]. Moreover, selecting a convenient AF depends on the case under analysis, and there is no standard procedure [25].

The current study found that the quality of the results strongly depends on the AF chosen. ReLU was the transfer function with the worst modelling predictions for forward as well as inverse problems. Nevertheless, this AF is the most widely used according to Ding et. al [23] for typical machine learning applications. Sigmoid and Sin provide good results for GVF, but their performance is not good for RVF. For the inverse problem, both AF provide good results, except for mid flow where Tanh gives a roughness value closer to those of GLUE. Sigmoid cannot be recommended for RVF.

The performance of Sin is not surprise according to Goodfellow et al. [26], who state that unpublished AF can have a similar performance to popular AF. Moreover, Hyperbolic Tangent (Tanh) is the transfer function with the best predictability for both forward and inverse problem. This AF has been widely adopted in PINN because it is infinitely differentiable, which is necessary to approximate the states of second or third-order partial differential equations governing different cases [2].

Neural network architecture

As in the case of the Activation Function, the number of layers and neurons per hidden layer plays an important role in a neural network performance [10, 25]. As stated in “Forward pinn: solving the differential equation” and “Inverse pinn” sections, the analysis of the number of layers and neurons is based on Tanh results for forward problem and inverse problem.

Tartakovsky et al. [2] explained that the number of layers and neurons depends on the smoothness of the output function. Moreover, the size of a neural network should be big enough to learn the mapping between inputs and outputs and small enough to be trained with the limited data available.

Forward problem: solving the differential equation

Case 1 and Case 3 (see Figs. 6a and c) have the most complicated water depth profile of the benchmark cases. Case 1 water profile is characterized by two peaks with two inflection points, and Case 3 presents a discontinuity due to a hydraulic jump. Both Cases have the neural networks with the biggest size. Furthermore, Case 2 has the smoothest water depth profile of all the cases (see Fig. 6b), having the second smallest neural network. The neural network dimensions of Cases 1, 2, and 3 agree with the information given above. Case 4 water depth profile (refer to Fig. 6d) has a smooth solution, so it is not surprise the size of its neural network being the smallest. Step-pool (Case 5), as expected, has a flow depth profile more complex than the benchmark cases due to the tumbling flow. The dimension of the neural network, when Tanh is used, is smaller than the previous most complex benchmark cases: Case 1 and Case 3.

The literature on PINN has shown that the number of hidden layers ranges from 7 to 9, while the number of neurons per hidden layer lies in the range of 20 to 120 neurons per hidden layer. The studied cases include the solution of forwarding problems using Burger’s Eq. (3), the solution of the Euler equation for high-speed flows [4], the use of KdV-Burger’s equation [10], and the fluid of blood in arteries applying a reduced form of Navier–Stokes equation [22]. Thus, the literature cases are representative of several phenomena in fluid flows. As mentioned in “Neural network architecture” section, the same pattern was obtained in this study, agreeing with the previously mentioned results except for Case 5. For Step-pool cases the resulting number of hidden layers is lower than the presented range, but the number of neurons per hidden layers is inside the range when Tanh is used. Nonetheless, if Sin or Sigmoid are used then the neural network dimension agrees with the found range of number of hidden layers and neurons per hidden layer.

Inverse problem

The available found literature for inverse problems is based on data from reference models or solvers: Raissi et al. [3] successfully found parameters of Burger’s equation and Navier Stokes equations in continuous time models and, Mao et al. [4] was able to find states of interest and parameters in high-speed aerodynamic flows. In this study, real staff gauges measurements for three flow magnitudes (low, mid, and high flow [19]) were used to obtain the roughness parameter.

The range of neural network architecture found in literature for inverse problem is highly variable: 3 to 9 hidden layers with 20 to 120 neurons per hidden layer [4, 5]. The architecture found in this research is inside the mentioned range. Furthermore, the number of hidden layers of the neural network varies depending on the flow magnitude while the number of neurons per hidden layer keeps constant. It has not been possible to find any reference with the application of PINN in Step-pool for inverse problem to compare the results. In that sense, this a first step in that direction.

PINN predictability

Case 1 and Case 2 show a GVF, meaning that the water depth never crosses the critical depth. Under these conditions the use of energy equation is allowed [17]. Case 1 and Case 2 has different flow types: Case 1 has subcritical flow, so there is low velocity and the flow is considered as tranquil. Case 2 has supercritical flow having high velocity and considered as rapid flow [44]. Under subcritical flow, all AF have a good performance to predict water depth profile except for ReLU. On the other hand, the prediction of supercritical flow was performed efficiently by all the AF.

Case 3 and Case 4 represent RVF where the energy equation cannot be applied. According to Brunner [17] in the case of rapidly varied flow, HEC-RAS uses the momentum equation for some instances such as hydraulic jump, low flow hydraulics at bridges, and stream junctions. In Case 4, PINN got a good answer when Tanh was used as an activation function, and the remaining activation functions got spurious discontinuities or non-physical answers. Indeed, PINN and HEC-RAS get the same answer because both solve the same equation. On the other hand, PINN was not able to predict the water depth profile in Case 3, producing a model with unsatisfactory performance. Even though HEC-RAS solves the momentum equation and PINN solves energy equation, PINN was able to predict the discontinuity in the water depth profile.

Case 5 represents a real system called Step-pool. For this system, besides having a complex profile, the cross sections are variable. Moreover, there is RVF at pools below the steps. Even through EF in Table 1 depicts a good fitting performance; Fig. 7a–c clearly shows small discrepancies in the water depth between PINN and HEC-RAS at some points, attributable to the different description of the cross-sectional geometry since in the proposed PINN all the cross sections geometry is contained in an equivalent cross section. In case of highly variable cross sections, it will be necessary to divide the reach into sub reaches each with a equivalent cross section and to implement continuity equations such as the ones used in Kissas et al. [22].

Case 3 and 5 contain hydraulic jumps in the water depth profiles. Case 3 has a sudden decrease of the slope, and Case 5 has tumbling flow. Despite being a more complex case, Case 5 PINN prediction are much better than Case 3. There are some reasons why this may happen. These are, first, the difference between yi and ys in Case 3 is 1.56 m while in Case 5 the difference ranges from 0.24 to 0.39 cm. The hydraulic jump in Case 3 is four times bigger than those in Case 5. Second, the roughness value in Case 3 is ten times smaller than in Case 5. Thus, the resistance to flow in Case 5 is bigger than in Case 3 meaning a higher energy dissipation besides the one in the hydraulic jump. It seems possible that as hydraulic jump gets higher due to a low flow resistance, the prediction of PINN get worse.

Systematic studies

Rate of convergence

Case 2 (Supercritical flow) has the highest rate of convergence. A possible reason of the different pattern may be produced by the smoothness of the solution. Indeed, as can be seen in Fig. 6b, the water depth in this Case do not have peaks, follow a descending pattern, and have small slopes. This may be the cause of the different convergence answers of PINN. Even though Case 4 has a sudden decrease of water depth (discontinuity), this case has a smooth solution. Thus, its high convergence rate is no surprise. Furthermore, the difference between both cases with discontinuities Case 4 and Case 3 (sudden increase of water depth) is the way in which HEC-RAS deals with the discontinuity. For a discontinuity like Case 3 HEC-RAS uses an alternative form of the momentum equation. However, for a Case 4 type of discontinuity HEC-RAS uses the same equation as ours: the equation of the energy (Eq. (1)).

The cases with a sudden increase in water depth (hydraulic jump) like Case 3 and Case 5 has the same rate of convergence 0.2. On the other hand, Case 1 (Subcritical flow) having a smooth solution with multiple peaks has a rate of convergence of 0.4. Thus, the effect of a discontinuity like a sudden increase of water depth affects the convergence by 50%.

Sensitivity test

Case 3 has the highest changes in the PINN model performance when boundary conditions are increased. The increasing of boundary depth could change the flow conditions in this case. Indeed, the boundary condition with supercritical flow could change to subcritical flow. In that case, there is no discontinuity (hydraulic jump), which explains the significant change in the performance of the method. On the other hand, the reduction of the value in the boundary condition might preserve the flow type, so the lack of sensitivity showed in Fig. 10c is justified.

Case 2 has the slowest change in the model fitting performance. Case 2 profile has an increasing slope, so the flow is going to be supercritical. The supercritical profile is smooth so any change in the BC is not going to affect the water depth in an important way.

Case 1 fitting performance is affected only when the BC change reaches the highest values. When BC water depth is increased the subcritical flow is preserved. However, it seems that when the increase in the BC reaches a certain value the prediction quality decreases. The reduction in the boundary could lead to a change in the flow conditions, so the oscillation present in − 4% in Fig. 10a could be justified.

The oscillations in performance when BC water depths are increased or decreased are to be expected in the real case. This case has the most complex geometry and water depth pattern. Thus, any change at the BC could have different effect in the predicted water depth.

Conclusions

In this research, the predictive performance of the Physical Informed Neural Network (PINN) has been tested for a forward and an inverse problem. Moreover, PINN is a tool where the physics of a system is used. Four open channels cases with different bed shapes and prismatic cross-sections have been proposed to test the approximation ability of PINN under different flow types: subcritical, supercritical, and mixed for forward problem. Moreover, a fifth case based on a Step-pool in the Quinuas river was also included to solve a forward and an inverse problem with PINN. In addition, PINN results for the forward problem were compared to HEC-RAS, while the inverse problem results were compared with the results of a previous study based on the GLUE methodology.

This study has provided several interesting results for forward and inverse problems. For forward problems, PINN has shown good approximation characteristics, when a high number of collocation points are used. The predictability of Step-pool water depth profile was considered good; however a close look to the profiles shows a slight difference between PINN and HEC-RAS probably as a result of the simplified cross-sectional information.

The activation function (AF) played an important role in the approximation performance on forward problems. The hyperbolic tangent (Tanh) ended up being the activation function with the best performance for forward and inverse problem, when there are a sufficiently large number of collocation points. Furthermore, Sin and Sigmoid did not provide adequate results for rapid flow cases in the forward problem, but these AFs provide good results in the inverse problem. ReLU had the worst results in all the studied cases.

The rate of convergence was higher in cases with smooth solutions, and poorer in cases with a sudden increase of water depth. The introduction of a perturbation at the boundary condition has different effects depending on the flow type at each Case.

Availability of data and materials

The dataset used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

AD:

Automatic differentiation

AS:

Analytical solution

AF:

Activation functions

BC:

Boundary conditions

CFD:

Computational Fluid Dynamics

EF:

Nash–Sutcliffe efficiency index

EDE:

Energy Differential Equation

GLUE:

Generalized Likelihood Uncertainty Estimation

GVF:

Gradually Varied Flow

MAE:

Mean Average Error

ML:

Machine Learning techniques

MSE:

Mean square error

MSEBC :

Mean square error at boundary conditions

MSEf :

Mean square error of the embedded physics

MSEG :

Mean square error of the longitudinal profile data

PDE:

Partial differential equations

PINN:

Physics-Informed Neural Networks

RMSE:

Root Mean Square Error

RVF:

Rapid Varying Flow

Tanh:

Hyperbolic Tangent

XSs:

Cross-sections

References

  1. Rao C, Sun H, Liu Y. Physics-informed deep learning for incompressible laminar flows. Theor Appl Mech Lett. 2020;10(3):207–12. https://doi.org/10.1016/j.taml.2020.01.039.

    Article  Google Scholar 

  2. Tartakovsky AM, Marrero CO, Perdikaris P, Tartakovsky GD, Barajas-Solano D. Physics-informed deep neural networks for learning parameters and constitutive relationships in subsurface flow problems. Water Resour Res. 2020;56(5):e2019WR026731.

    Article  Google Scholar 

  3. Raissi M, Perdikaris P, Karniadakis GE. Physics informed deep learning (part i): Data-driven solutions of nonlinear partial differential equations. arXiv Prepr arXiv171110561. 2017. http://arxiv.org/abs/1711.10561. (Accessed 21 Aug 2020)

  4. Mao Z, Jagtap AD, Karniadakis GE. Physics-informed neural networks for high-speed flows. Comput Methods Appl Mech Eng. 2020

  5. Raissi M, Perdikaris P, Karniadakis GE. Physics informed deep learning (Part ii): Data-driven discovery of nonlinear partial differential equations. arXiv Prepr arXiv171110566v1. 2017;

  6. He Q, Tartakovsky AM. Physics-informed neural network method for forward and backward advection-dispersion equations. Water Resour Res. 2021;57(7):e2020WR029479.

    Article  Google Scholar 

  7. Güneş Baydin A, Pearlmutter BA, Andreyevich Radul A, Mark SJ. Automatic differentiation in machine learning: a survey. J Mach Learn Res. 2018;18:1–43.

    MathSciNet  MATH  Google Scholar 

  8. Raissi M, Perdikaris P, Karniadakis GE. Multistep neural networks for data-driven discovery of nonlinear dynamical systems. arXiv Prepr arXiv180101236. 2018. http://arxiv.org/abs/1801.01236. (Accessed 1 Sep 2020)

  9. Wang R, Kashinath K, Mustafa M, Albert A, Yu R. Towards physics-informed deep learning for turbulent flow prediction. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery\& Data Mining. 2020; 1457–66.

  10. Guo Y, Cao X, Liu B, Gao M. Solving partial differential equations using deep learning and physical constraintsts. Appl Sci. 2020;10(17):5917.

    Article  Google Scholar 

  11. Papanicolaou AN, Bdour A, Wicklein E. One-dimensional hydrodynamic/sediment transport model applicable to steep mountain streams. J Hydraul Res. 2004;42(4):357–75.

    Article  Google Scholar 

  12. Sart C, Baume J-P, Malaterre P-O, Guinot V. Adaptation of Preissmann’s scheme for transcritical open channel flows. J Hydraul Res. 2010;48(4):428–40.

    Article  Google Scholar 

  13. Berger RC, Stockstill RL. Finite-element model for high-velocity channels. J Hydraul Eng. 1995;121(10):710–6.

    Article  Google Scholar 

  14. Maxwell AR, Papanicolaou AN. Step-pool morphology in high-gradient streams. Int J Sediment Res. 2001;16(3):380–90.

    Google Scholar 

  15. MacFarlane WA, Wohl E. Influence of step composition on step geometry and flow resistance in step-pool streams of the Washington Cascades. Water Resour Res. 2003. https://doi.org/10.1029/2001WR001238.

    Article  Google Scholar 

  16. Chin A, Wohl E. Toward a theory for step pools in stream channels. Prog Phys Geogr. 2005;29(3):275–96.

    Article  Google Scholar 

  17. Brunner G. HEC RAS, river analysis system hydraulic reference manual. 2021.

  18. Artichowicz W, Mikos-Studnicka P. Comparison of average energy slope estimation formulas for one-dimensional steady gradually varied flow. Arch Hydro-Engineering Environ Mech. 2014;61(3–4):89–109.

    Article  Google Scholar 

  19. Cedillo S, Sánchez-Cordero E, Timbe L, Samaniego E, Alvarado A. Patterns of Difference between Physical and 1-D Calibrated Effective Roughness Parameters in Mountain Rivers. Water. 2021; 13(22): 3202. https://www.mdpi.com/2073-4441/13/22/3202/htm. (Accessed 12 Nov 2021)

  20. Cedillo S, Sánchez-Cordero E, Timbe L, Samaniego E, Alvarado A. Resistance analysis of morphologies in headwater mountain streams. Water. 2021;13(16):2207.

    Article  Google Scholar 

  21. Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, et al. TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems. arXiv Prepr arXiv160304467. 2016. http://arxiv.org/abs/1603.04467. (Accessed 1 Sep 2020)

  22. Kissas G, Yang Y, Hwuang E, Witschey WR, Detre JA, Perdikaris P. Machine learning in cardiovascular flows modeling: predicting arterial blood pressure from non-invasive 4D flow MRI data using physics-informed neural networks. Comput Methods Appl Mech Eng. 2020;358:112623.

    Article  MathSciNet  Google Scholar 

  23. Ding B, Qian H, Zhou J. Activation functions and their characteristics in deep neural networks. In: 2018 Chinese control and decision conference (CCDC). Institute of Electrical and Electronics Engineers Inc.; 2018. p. 1836–41.

  24. Nwankpa C, Ijomah W, Gachagan A, Marshall S. Activation Functions: Comparison of trends in Practice and Research for Deep Learning. arXiv Prepr arXiv181103378. 2018. https://arxiv.org/abs/1811.03378v1. (Accessed 9 Aug 2021)

  25. Sharma S, Sharma S, Athaiya A. Activation functions in neural networks. Int J Eng Appl Sci Technol. 2020;04(12):310–6.

    Google Scholar 

  26. Goodfellow I, Bengio Y, Courville A. Deep Learning. MIT Press; 2016. https://www.deeplearningbook.org/. (Accessed 23 Jul 2021)

  27. Lovric M. International Encyclopedia of Statistical Science. Miodrag L, editor. International Encyclopedia of Statistical Science. Berlin Heidelberg: Springer; 2011. p.1673.

  28. Dodge Y. The Concise Encyclopedia of Statistics The Concise Encyclopedia of Statistics. Berlin Heidelberg: Springer Science and Business Media; 2008.

    MATH  Google Scholar 

  29. Wohl E. Uncertainty in flood estimates associated with roughness coefficient. J Hydraul Eng. 1998;124(2):219–23.

    Article  Google Scholar 

  30. Bhola PK, Leandro J, Disse M. Reducing uncertainties in flood inundation outputs of a two-dimensional hydrodynamic model by constraining roughness. Nat Hazards Earth Syst Sci. 2019;19(7):1445–57.

    Article  Google Scholar 

  31. Papaioannou G, Vasiliades L, Loukas A, Aronica GT. Probabilistic flood inundation mapping at ungauged streams due to roughness coefficient uncertainty in hydraulic modelling. Adv Geosci. 2017;44:23–34.

    Article  Google Scholar 

  32. Horritt MS, Bates PD. Evaluation of 1D and 2D numerical models for predicting river flood inundation. J Hydrol. 2002;268(1–4):87–99.

    Article  Google Scholar 

  33. Marriott M, Featherstone RE, Nalluri C. Nalluri And Featherstone’s Civil Engineering Hydraulics: Essential Theory with Worked Examples. Hoboken: Wiley; 2016.

    Google Scholar 

  34. Willmott CJ, Matsuura K. Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance. Clim Res. 2005;30(1):79–82.

    Article  Google Scholar 

  35. Chai T, Draxler RR. Root mean square error (RMSE) or mean absolute error (MAE)? -Arguments against avoiding RMSE in the literature. Geosci Model Dev. 2014;7(3):1247–50.

    Article  Google Scholar 

  36. Merz R, Blöschl G. Regionalisation of catchment model parameters. J Hydrol. 2004;287(1–4):95–123.

    Article  Google Scholar 

  37. McCuen RH, Knight Z, Cutter AG. Evaluation of the Nash-Sutcliffe Efficiency Index. J Hydrol Eng. 2006;11(6):597–602.

    Article  Google Scholar 

  38. Nayak PC, Venkatesh B, Krishna B, Jain SK. Rainfall-runoff modeling using conceptual, data driven, and wavelet based computing approach. J Hydrol. 2013;493:57–67.

    Article  Google Scholar 

  39. Ritter A, Muñoz-Carpena R. Performance evaluation of hydrological models: Statistical significance for reducing subjectivity in goodness-of-fit assessments. J Hydrol. 2013;480:33–45.

    Article  Google Scholar 

  40. Jacquemin T, Bordas SPA. A unified algorithm for the selection of collocation stencils for convex, concave, and singular problems. Int J Numer Methods Eng. 2021;122(16):4292–312.

    Article  MathSciNet  Google Scholar 

  41. Fish J, Belytschko T. A first course in finite elements. A first course in finite elements. 2007.

  42. Glorot X, Bengio Y. Understanding the difficulty of training deep feedforward neural networks. J Mach Learn Res. 2010;9(May):249–56.

    Google Scholar 

  43. Sibi P, Allwyn Jones S, Siddarth P. Analysis of different activation functions using back propagation neural networks. J Theor Appl Inf Technol. 2013;47(3):1264–8.

    Google Scholar 

  44. Chow VT. Open-Channel Hydraulics. Science. New York: McGraw-Hill civil engineering series; 1959, 680.

Download references

Acknowledgements

Not applicable.

Funding

This research was funded by the Research Directorate of the University of Cuenca and developed within the framework of the project “Towards a sound mathematical description of the dispersion of pollutants in mountain rivers”.

Author information

Authors and Affiliations

Authors

Contributions

SC worked in the data acquisition, analysis and interpretation of data from PINN and HEC RAS. Moreover, SC works actively in the first version of the manuscript including its structure and posterior correction. A-GN developed the PINN coding and run all the studied cases: forward and inverse models. ES-C works in the data analysis and interpretation of PINN results providing substantial insights to improve PINN architecture. Furthermore, ES-C worked int the manuscript structure and corrected the first version of manuscript. LT design the field experiments, interpretation of HEC-RAS data and worked actively in the final version of the manuscript. ES work on the conception of the work as well as in the structure of the manuscript. Moreover, ES collaborated to improve PINN architecture and worked actively in the final version of the manuscript. AA worked on data analysis/ interpretation of PINN results. Moreover, AA administrated the research project funding this research, and revised the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Sebastián Cedillo.

Ethics declarations

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cedillo, S., Núñez, AG., Sánchez-Cordero, E. et al. Physics-Informed Neural Network water surface predictability for 1D steady-state open channel cases with different flow types and complex bed profile shapes. Adv. Model. and Simul. in Eng. Sci. 9, 10 (2022). https://doi.org/10.1186/s40323-022-00226-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s40323-022-00226-8

Keywords