Particle Swarm Optimization Neural Network for Flow Prediction in Vegetative Channel

1 Introduction

Channel vegetation refers to flora that typically are emergent aquatic plants or herbs, trees, and shrubs that thrive in and close to water. They may be submerged completely or non-submerged (emergent). They can have different densities, height of submergence, stem flexibility, stem geometry, surface characteristics, and spacing. These characteristics influence the flow and morphology of a channel in different ways. Submerged and emergent vegetation cause different velocity profiles, governed by their height and flexibility, in a channel.

Literature is replete with numerous works on flow–vegetation interactions. Huang and Nanson [12] found that vegetation affected channel hydraulic geometry exponents. Other studies such as those of McKenney et al. [22] and Malkinson [21] have found the effects of riparian vegetation on channel properties. These properties include the cross-sectional characteristics of streams, channel morphogenesis, and channel widening. Channel mobility has been found to decrease with increase in vegetation density [15]. Afzalimehr and Dey [1] studied the deviation of the velocity profile in the outer layer over a gravel bed. They reported that the deviation with vegetation cover on the walls is considerably larger than without vegetation cover on the walls. Studies have also shown that vegetation spacing affects turbulence by changing the three-dimensional flow patterns [29]. Experiments in laboratory flumes have been carried out to quantify the flow–vegetation interactions by various researchers [2, 32]. Natural vegetated field measurements have been performed by Green [11]. Detailed numerical simulations of flow through vegetation were also performed by Choi and Kang [5] and Neary [24]. Several empirical [17] and theoretical relations [25, 30] have been proposed to describe the flow–vegetation interactions. Huthoff et al. [14] have ascertained that because of simplicity, empirical equations have better field applicability than analytical ones. A common problem of empirical equations is that they are only applicable for restricted circumstances – those in which equations have already been formulated. Aside from empirical techniques, computational fluid dynamics models have been used to solve the Navier–Stokes equations in specified domains [5, 24]. Due to computational limitations, vegetation characteristics, most importantly, the vegetation geometry, have to be kept simple enough. This leads to idealizations that differ much from actual vegetation characteristics.

Many river hydraulics problems are characterized by a lack of proper understanding of such natural phenomena. This has led modelers to use soft computing techniques for tasks such as non-linear data modeling. These techniques make it possible to predict parameters based on significant recognizable variables. Over the last two decades, soft computing methods have caused a growing interest in hydraulics and river modeling applications. They have been used as powerful alternative computational tools for modeling tasks. Recent developments in the paradigm of bio-inspired computing introduced a variety of optimization algorithms. One such algorithm is particle swarm optimization (PSO) by Kennedy and Eberhart [16]. PSO is primarily a non-classical and evolutionary optimization algorithm. It continuously tries to improve the solution by browsing a group of potential solutions. These individuals are called “particles,” and they move in the search space of solutions. Decisions about the quality of solutions are based on a fitness criterion specified, usually the sum of squared errors. Each particle’s movement is inspired from two “good” solutions – its own personal best yet and the overall best solution of the population. A rule imparts velocity to each particle in the population and is influenced by these two good solutions. The velocity rule also incorporates random coefficients to enhance exploration and introduce non-determinism. It is expected that although moving toward the best solution, each particle traverses the space extensively and eventually toward the best solution. Hopping from one solution to another rather than jumping straight to the best is ensured by restraints on particle velocity. The population is commonly referred to as the “swarm.” The major advantage exploited is that the search space may not be differentiable, but it must be continuous in the domain described. The rule imparting velocity to the particles is a simple algebraic equation, and each individual particle has a simple function. Due to emergence, however, swarming leads to the discovery of solutions to complex problems when the entire swarm works for a common goal. Because the parameter determination of an artificial neural network (ANN) is essentially an optimization task, PSO can be used to determine network weights. The optimization task is to minimize the errors in the predictions of the model. PSO has been used extensively as an alternative optimization algorithm in the past. PSO was applied for the optimization of gravity dam and sluice gate by Wu et al. [33]. Santos et al. [27] have used PSO for physically based erosion model. A PSO-based simulation–optimization model for the solution of groundwater management problems was proposed by Gaur et al. [10]. ANN models have been used in the past for numerous problems requiring significant accuracy in prediction and classification tasks. ANN serves to be an implicit model that manipulates the input(s) using several network parameters – weights of connecting links and transfer functions operating at the node level. Conventional ANN setups have been used in several instances in the past. However, there are certain difficulties that ANNs are prone to when used with back-propagation (BP) training algorithm. The selection of network architecture and its efficient training procedure are major obstacles for any ANN-based model. From the time complexity analysis, El-Telbany and El-Karmi [7] found that both PSO and BP algorithms converge approximately to O(N³). They also derived a conclusion on the predictability of the neural network trained by BP algorithm over the test data. According to them, the PSO algorithm performed better than the BP algorithm. In a hybrid ANN–PSO combination, while the ANN handles the non-linearity, PSO is used to optimize the ANN parameters for an accurate fit. Rong et al. [26] have used the ANN–PSO approach to diagnose hydraulic turbine vibration fault. Li et al. [19] have used ANN–PSO to optimize the dam structure.

The primary objective of the present work is to develop the flow prediction model of a submerged vegetative channel based on a PSO–ANN approach. In the ANN–PSO hybrid combination used, modeling is accomplished using an ANN. PSO is complementarily used for assigning optimal weights to the ANN. An ANN is proposed with optimal parameters, and weights are then adjusted. This is done by a PSO run with several numbers of particles exploring the solution space by continuously monitoring and moving toward the best obtained positions in the solutions space yet. The vegetation characteristics and other flow characteristics are presented to the thereby trained network to assess its predictability. The available empirical equations are applied to the same data to compare the performance and reliability of the models. Gradient descent algorithms have also been applied for the same data sets. Performance is assessed by several measures including Nash–Sutcliffe model coefficient and index of agreement criteria.

2 Functional Analysis of the Flow–Vegetation Interaction

A functional form of a physical system describing its dependent and independent variables is necessary for modeling system behavior. Yen [34] has analyzed several equations in terms of their dependent and independent variables. The physical modeling of channel flow-through vegetation is very complex. According to Lopez and Garcia [20], it is a function of many variables: fluid properties, flow properties, vegetation characteristics, and channel characteristics. A closer look (Table 1) on the existing equations shows that flow–vegetation interactions can be described by

Table 1

Available Descriptors for Flow Velocity in a Vegetative Channel.

Author(s)	Equation(s)
Stone and Shen [29]
Van Velzen et al. [31]
Baptist et al. [3]
Huthoff [13]

where u is the mean velocity, h is the flow depth, k is the height of the vegetation, i is the channel slope, D is the diameter of cylindrical vegetation, m is the number of cylinders per square meter horizontal area, and C_d is the non-dimensional drag coefficient. Equation (1) is used for homogeneous vegetation of fixed height and diameter. Flow is assumed to be steady and uniform. Assuming a wide channel, the sidewall effects are neglected [4]. This present work aims in predicting the flow velocity of a submerged vegetated channel; thus, u can be made a dependent variable that depends on other various factors as shown in Equation (1). In line with this, Equation (1) can be written as

The objective here is to construct a neural network model that approximates an unknown input–output mapping of Equation (2) based on given real observations. The goal, however, is not to provide an exact fit to the data. The primary aim is to develop a model that captures the underlying relationship so that it can be used to predict the output at some future observation of the input.

3 Neural Network Modeling

ANNs have been in use for decades as a non-linear modeling tool. An ANN is basically a group of connected neurons having transfer functions together with its various connection weights. The transfer functions within a neuron are usually non-linear sigmoid functions. They allow an accurate description of non-linearity present in the data. Conventionally, BP is used to train ANNs and has been found to be very successful. However, certain aspects of such conventional gradient descent techniques make training of ANN difficult in certain situations. They require that the solution space be differentiable. Problems of river hydraulics in general are complicated owing to many interdependent variables. The intricate dependence of the behavior of a hydraulic system is sometimes very sensitive to some variables but is not significantly affected by other variables. Because a clear understanding of the complex interdependence is lacking, an assumption of differentiability of the error hypersurface might be superfluous at times. In the problem of vegetated flow, interactions occur among vegetation, bed, and flow, causing changes in the flow pattern. Besides, the flow is characterized by turbulence at the microscopic scale. The gradient descent methods optimize by moving the ANN weights over a fixed, decreasing profile. They are therefore bound to miss out combinations of network weights lying in a region that does not fall in this fixed path. These missed out combinations are potential candidates and may lead to better solutions of an optimization problem, here, the training of network. This is synonymous with gradient methods missing out global optima and being trapped in a local optimum instead due to the profile of the error hypersurface. More recent algorithms from the paradigm of evolutionary computing have made it possible to overcome such limitations of training algorithms. PSO is one such robust non-linear optimization algorithm that is inspired by another natural phenomenon – the social behavior of swarms. Here, individuals operate for a common goal, using each other’s experience and exploration capabilities to reach the best desired locations. Other methods offered by soft computing techniques include genetic algorithm (GA) and ant-colony optimization (ACO). The methods, although inspired from nature, are different fundamentally. Both PSO and GA use generations of individuals, but they use disparate algorithms for generating new individuals using the older ones. GA uses performance criteria to sort out “good” individuals from previous generations, cross their features, and reach a new, possibly better solution. PSO, meanwhile, uses proximities of other individuals of the swarm to help in deciding the position of the current individual. In that sense, the experience of an individual gets accumulated generation after generation in PSO. Although the population may even lose potentially good solutions in the probabilistic selection procedures in GA, a PSO particle is never eliminated. Thus, PSO overcomes some difficulties encountered in GA-based optimization. Another successive technique over which PSO has an advantage is the ACO. As has been stated by Selvi and Umarani [28], ACO is more useful for applications that require reaching crisp and definitive results. They also concluded that PSO is a better choice for problems where targets are fuzzy in nature.

Finding the weights of an ANN is essentially an optimization problem. The data set provided to the ANN forms equations with weights as parameters that have to be satisfied. For the PSO, cost function is the net error corresponding to a set of weights in the N-dimensional search space, N being the number of weights. In a typical PSO iteration, each potential solution is “flown” through the N-dimensional hyperspace of possible solutions. There is an inclination to move toward two current optimal solutions – their own personal bests yet and the global best yet. Owing to the randomness and attraction toward two different optimal values, a particle hops the space, and this is how it encounters still better solutions than recognized in the previous iterations. The particles are initialized with random solutions, and after a certain number of iterations, they settle or converge on a common solution. This is the best solution found yet by the entire swarm. To ensure simultaneous convergence in all dimensions of the solution or search space, a high number of particles and iterations are crucial for a PSO run. This is to carry out extensive search in the huge search space. Any point in the search space is a potential solution to the algorithm. The error associated with any such point forms a d-dimensional hypersurface, with d being the number parameters to be optimized. Because the nature of the actual error position hypersurface is unknown, the situation may become oblivious when it comes to deciding PSO parameters. PSO parameters include the number of particles, inertia weights, and the acceleration coefficients – cognitive as well as social. There is no prescribed general method to identify proper parameters of a PSO algorithm. This has been addressed by a repeated trial procedure. Different combinations of the various parameters were used, and algorithm performance was assessed in terms of the accuracy of the solution (model efficiency coefficient). Moreover, a common issue with virtually any optimization problem is being trapped in local minima rather in the global minimum in the search space. Although the PSO does not count on the differentiability of the search space, when the swarm nears a local minimum, it is itself very likely to be trapped in the same. This does happen and is one of the major drawbacks of the PSO approach. This problem was dealt by Evers [8] in the development of a regrouping mechanism. Here, as soon as the swarm is found to stagnate in any minima, it is randomly regrouped in a smaller search space. The previous global best solution is used as the center of the new search space. This allows for a more robust algorithm and works well in eliminating non-optimal solutions.

4 Results and Discussions

A hybrid ANN–PSO model was adopted to describe the behavior of the system at hand. The entire modeling and analysis has been done using the neural network toolbox of the MATLAB^® software (The MathWorks Inc., Natick, MA, USA). Galema [9] documented the comprehensive database of different types of vegetations with hydraulic properties from different sources. Table 2 shows the ranges of the observations used in the present work. The total number of observations used in the modeling is 445. For the development of ANN model, 75% of the observations have been assigned as training sets and 25% have been assigned for testing. Not all data sources are cited in the reference list of this article because they could be found in the work of Galema [9]. The optimal parameters for the ANN design and the PSO algorithm were conclusively reached by a hit-and-trial-based analysis. Many iterations of the program were carried out to reach to an optimum value of the ANN design parameters – the number of layers and number of neurons in a layer. For an ANN, the number of neurons is unknown and has to be chosen carefully. Although the presence of too many layers, neurons, and a complicated structure yield a network that can easily get overtrained, thereby “memorizing” outputs, too few neurons, meanwhile, could lead to a network that cannot describe the variability in the data presented. It was observed that the network easily memorized outputs when it has more than one hidden layer. Hence, an appropriate number of neurons had to be reached, which was identified by making several runs of the training algorithm with a different number of neurons in a single hidden layer. This process is illustrated in Figure 1. No algorithmic changes were made in these runs except for the number of neurons. Second, the transfer functions of the ANN were decided using a similar approach. Different combinations of the transfer functions – tan sigmoid and log sigmoid – were used for the hidden layer. Finally, the combination that provided the best correlation coefficient values was adopted. The log sigmoid and tan sigmoid functions were adopted for hidden and the input layers, respectively. A linear transfer function was used for the output layer.

Figure 1

Ascertaining the Optimal Number of Neurons in Hidden Layer.

Twenty neurons were adopted for the ANN because they gave the best correlation coefficient compared with a different number of neurons. Several runs were attempted, keeping the various parameters constant so that the influence of number of neurons alone could be assessed. With 20 neurons, the number of weights were 140, including 120 input weights and another 20 layer weights. Once the network design is decided, the number of parameters (weights) of the ANN is known. The problem then was one of optimization – deciding the best combination of ANN weights that explains model behavior appropriately. PSO was adopted because, compared with conventional BP algorithms, it does not make assumptions about the search space or require a differentiable search space to work. PSO is a well-established heuristic optimization technique and has shown to be of comparable competence with conventional techniques. The PSO algorithm itself has a variety of parameters to be adjusted. The various parameters of the algorithm are search-space-specific, and there is no general rule to reach a conclusion. Therefore, the parameters of PSO were reached again by several trials with different combinations. The parameters that need to be decided include number of particles, inertia weight of particles, and acceleration coefficients – cognitive and social. References were used to reach a suitable estimate of PSO parameters. An analysis of the particle swarm algorithm was carried out by Clerc and Kennedy [6], and a way of finding a set of coefficients to control the system’s convergence tendencies was established. The equivalent parameters of Clerc and Kennedy [6] were used as suggested by Evers [8]. The inertial weight was constant and kept as 0.72984, whereas both the social and the cognitive acceleration terms were 1.4 and 1.5, respectively. The cognitive coefficient was decreased nominally from the recommended value to control the convergence characteristics of the moving swarm in a profitable manner. Moreover, for the regrouping of the swarm, the stagnation detection technique used a threshold radius of the swarm. As suggested by Evers [8], the radius of the swarm was kept at 1.1×10⁴ because it allowed for a required amount of solution refinement before regrouping and produced meaningful measures of uncertainty. Finally, the number of iterations was increased repeatedly from a base value of 1000. At each trial, it was observed that the performance of the training algorithm increased initially significantly. By reaching the value of 22,000, the performance of PSO stagnated with an asymptotic nature, as shown in Figure 2, where the error goal is absolute zero.

Figure 2

Performance of the Neural Network.

A regrouping PSO algorithm adopted, as proposed by Evers [8], enables the avoidance of stagnation in the local minima in a search space. As soon as stagnation is detected, the set of particles are regrouped again around the previous swarm minima. This increases the likelihood of the global minima being detected. Stagnation is detected when the particles are found to occupy a small hypersphere in the search space. The total number of iterations in a PSO indicates the number of times particles are moved in the search space for finding “good” solutions, while a maximum number of PSO iterations were also specified per swarm regrouping. For the current 140-dimensional search space, a significantly high number of iterations were adopted. This allowed the particles to navigate the search space intensively in search for better solutions.

The final architecture of the neural net used in the analysis is 6-20-1 and is illustrated in Figure 3. Model prediction has been shown in the Figure 4A and B for the training and test sets, respectively. As seen in Figure 4, the model predicts the phenomena well. For the illustration of gradient-based approaches, two standard training algorithms have been used – the Levenberg–Marquardt (LM) optimization and the BFGS quasi-Newton method. In the present work, the formulae listed in the Table 1 have been also tested to quantify their prediction capability. The efficiency criteria chosen in the present work to ascertain model prediction are root-mean-square deviation (RMSD), mean absolute percentage error (MAPE), coefficient of determination (R²), Nash–Sutcliffe efficiency (E), and index of agreement (I_d). RMSE is used as a standard measure of the differences between the model-predicted and observed values. It serves well in depicting the predictive power of a model. Meanwhile, MAPE is a measure of the accuracy of a method and expresses accuracy as percentage. It is known that RMSE is scale dependent, and hence, other parameters such as R² are used for assessing model performance. A major disadvantage of MAPE is that a few data points might distort a comparison. R² can also be expressed as the squared ratio between the covariance and the multiplied standard deviations of the observed and predicted values. Therefore, it estimates the combined dispersion against the single dispersion of the observed and predicted series. A major drawback of R² is that it quantifies the dispersion only. The efficiency E proposed by Nash and Sutcliffe [23] is defined as one less the sum of the absolute squared differences between the predicted and the observed values normalized by the variance of the observed values. The range of E lies between 1.0 (perfect fit) and –∞. An efficiency <0 indicates that the mean value of the observed series would have been a better predictor than the model. The most significant disadvantage of the Nash–Sutcliffe efficiency is the fact that the differences between the observed and the predicted values are calculated as squared values. As a result, larger values in a time series are strongly overestimated and lower values are neglected [18]. The index of agreement I_d was proposed by Legates and McCabe [18] to overcome the insensitivity of E and R² to the differences in the observed and predicted means and variances. The index of agreement represents the ratio of the mean square error and the potential error. The performance analysis based on these coefficients is listed in Table 3. Among the empirical predictors, the formula of Baptist et al. [3] predicts well when compared with all the efficiency criteria (R², E, and I_d). The performance of the gradient-based methods (ANN–LM and ANN–BFGS) was found to be comparable with those of the PSO algorithm. However, the results of training the algorithm with PSO gave superior predictive models, as reflected by model performance indicators mentioned in Table 3.

Figure 3

Optimal ANN [6-20-1].

Figure 4

Results of Neural Network Model: (A) Training Sets and (B) Test Sets.

5 Conclusion

Flow–vegetation interactions are central to many problems of practical interest to hydraulic engineers including flood risk studies, sediment transport studies, and the analysis of the hydraulic performance of river restoration schemes. Existing predictors are too complex and limited in terms of their applicability. Based on a large database, the present work formulates a alternative hybrid neural network model of mean velocity in a vegetative channel comprising all the relevant parameters. This way, a qualitative effect of all type of parameters can be incorporated in the prediction of flow velocity in a vegetative channel. The generalization capacity of the model is superior, considering the multiple interdependent parameters affecting the flow–vegetation interactions. The non-linear and non-parametric nature of the neural network model is desirable for field applications. A comparative analysis of empirical predictors validates that the present model performs better than the existing empirical predictors.