Abstract
Differential evolution (DE) is a population-based algorithm using Darwinian and Mendel principles to find out an optimal solution to difficult problems. In this work, the dynamics of the DE algorithm are modeled by using a longitudinal social network. Because a population of the DE algorithm is improved in generations, each generation of DE algorithm is represented by one short-interval network. Each short-interval network is created by individuals contributing to population improvement. On the basis of this model, a new parent selection in the mutation operation is presented and a well-known benchmark set CEC 2013 Special Session on Real-Parameter Optimization (including 28 functions) is used to evaluate the performance of the proposed algorithm.
1 Introduction
Differential evolution (DE) is an efficient algorithm for optimization introduced by Price and Storn in 1997 [10]. DE is easy to use, and its convergence properties are considered to be very good. Despite these facts, DE also has several disadvantages: its convergence is unstable and it easily drops into the local extreme [14]. To eliminate or even remove these bottlenecks is the goal of many researchers from all over the world.
Brest et al. described DE using self-adaptive control parameters (jDE), where each individual is extended with its own control parameters Fi (scale parameter) and CRi (crossover rate) [2]. Wang et al. proposed the DE with composite trial vector generation strategies and control parameters [13], where three mutation strategies are combined with three different control parameter settings to create three trial vectors. Zhou et al. introduced the DE with an intersect mutation operator called IMDE [17]. In this algorithm, the populations of individuals are divided into better and worse parts according to their objective function value. Zhou et al. also presented a new DE with a hybrid mutation operator and self-adapting control parameters for global optimization (HSDE), where each individual is enhanced by its own scale parameter Fi and crossover rate CRi as in the case of jDE. Then, for each target vector, a mutation vector is created such that five mutually different parents are selected randomly. If an objective function value of a target vector is better than an objective function value of the first two parents, the DE/rand/1/bin strategy is applied to generate a mutation vector; otherwise, DE/current-to-best/1/bin is applied.
Most of above-mentioned researches deal with three ways on how to improve DE. The first one is to propose more efficient mutation operators, the second one is focused on the control parameter settings, and the third method deals with a process of parent selection in a mutation operation. In this work, a longitudinal social network (LSN) is used to model the DE dynamics. The main motivation is to better understand relationships between individuals and use of this knowledge to improve the DE performance.
The idea to model evolutionary algorithm dynamics using a complex network (CN) was presented by Zelinka et al. in 2010, where the authors investigated a possibility to visualize the dynamics of evolutionary algorithms by using CNs [16]. One year later, Zelinka et al. visualized evolutionary algorithm dynamics by using CNs; moreover, the authors visualized the dynamics of the created CNs using the coupled map lattices [15]. In 2014, Davendra et al. analyzed the CNs generated by Discrete Self-Organizing Migrating Algorithm, where a permutation flow-shop scheduling problem with blocking constraint had been used as the test problem [4].
The last mentioned publications [4, 15, 16] dealt with possibility to create CNs by evolutionary algorithm dynamics and with the analysis of such generated CNs. However, we have found no research where the dynamics of an evolutionary algorithm is modeled by LSN, and the results of this analysis are used to improve the evolutionary algorithm’s performance. In this work, an LSN is used to model the dynamics of the DE algorithm. An LSN consists of the snapshots of the network capturing the state of the network in different points of time. These snapshots are called short-interval networks (SINs). In this work, each generation is modeled by one SIN to capture positive relationships between individuals in a population established in this generation. On the basis of this model, a new parent selection in the mutation operation is presented, where the parents are selected on the basis of the strength of nodes representing these individuals in the LSN.
The rest of the paper is organized as follows. The DE algorithm is described in Section 1.1. In Section 2, modeling of the DE dynamics by LSN is clarified and the novel principle of parent selection in the mutation operation is described. The results provided by the original version as well as by the novel DE algorithm are provided in Sections 3 and 4.
1.1 Differential Evolution
DE works with a population of NP individuals
The mutation operation is mathematically defined by the following equation:
where F is the scale parameter with a typical value between 0.4 and 0.95 [7].
The crossover operation is then described by the following equation:
where CR∈[0, 1], rn(i)∈{1, …, D} is an integer selected randomly with the uniform distribution ensuring that a trial vector
2 LSN Created by DE Dynamics
In Refs. [15, 16], Zelinka et al. described the principle of CN creation by selected evolutionary algorithm dynamics. This principle accepts the philosophy that individuals of a population move from worse to better positions (similarly to individuals in swarm algorithms), which means that theoretically, a population consists of the same individuals just changing their positions. More precisely, a new offspring creation is considered to be just an activation of a target vector
As it was mentioned in Section 1, in this work, LSNs are used to model the dynamics of the DE algorithm. LSN consists of a number of different snapshots, so called SINs, capturing a state of the social network at different points in time. LSNs have been successfully used in different areas of research to better understand the difficult processes in various contexts [12]; for example, smoking behavior among adolescents in British schools [8], dynamic spread of happiness in a large social network [5], the spread of obesity in a large social network [3], virus spread in computer networks [9], etc.
In this work, each generation of the DE algorithm is modeled by one SIN. Each SIN captures only positive relationships between individuals in a population. In the case that a mutation vector
2.1 Parent Selection on the Basis of Node Strength
In the previous section, an LSN creation on the basis of DE dynamics has been described. As mentioned, for each generation, one SIN is created. Thanks to this mechanism, positive relationships between individuals established in the generation are captured. Each SIN is created by the nodes represented only by individuals contributing to improvement of a population. This means that the number of individuals in two consecutive SINs can be different. The number of arcs leading to the node representing a solution vector
Generally, a degree of the node i is the number of edges connecting this node with the other nodes, which is defined by the following equation:
where aij is the element of an adjacency matrix defined as 1 if a node i is connected to a node j, and 0 otherwise. In oriented graphs, in-degree is then defined as
For the weighted networks, the node strength si is defined as
where wij is an element of a weighted adjacency matrix and wij>0 if a node i is connected to a node j. The value of wij represents the weight of the edge. For weighted oriented networks, in-strength
In 2011, Uddin et al. introduced a time scale degree centrality, taking into consideration the presence as well as duration of links among actors within a network. The work of Uddin et al. from 2015 [12] proposed three new measures to quantify the dynamicity of LSNs: positional dynamicity, participation dynamicity, and complete dynamicity. Positional dynamicity represents changes of actor positions in different SINs relative to positions of these actors in the aggregated network. Participation dynamicity illustrates the actors changing network participation in any two consecutive SINs. Finally, the complete dynamicity of an actor is given by the summation of its positional and participation dynamicity. These measures seem to be very promising in the future research of the LSN generated by evolutionary algorithm dynamics.
From Eq. (1), it is obvious that the out-degree of each node in SIN will be 0 or 3. On the other hand, the in-degree of each node will be different because one solution vector can be used to create a mutation vector more than once. A node with the highest in-degree in the SIN representing a generation G will represent the solution vector, which has been used to create a new offspring in the generation G most often. On the other hand, the out-degree 3 of a node means that the individual represented by this node has been replaced by better offspring and its properties might positively influence a population evolution in the next generation.
In the following experiment, the mutation operator has been modified such that one solution vector can be selected simultaneously as the solution vectors
- (7)
- (8)
- (9)
In the case of Eqs. (7) and (8), two arcs would lead from the node representing a target vector
3 Experimental Results
The algorithm is implemented in C#, Microsoft.NET Framework 4.5.1, and run on a computer with CPU Intel(R) Xeon(R) 2.83 GHz. To evaluate the performance of the presented algorithm, 28 benchmark functions from the CEC 2013 [6] have been used.
In our work, the number of problem dimensions is set to 30 for all 28 test functions. The detailed parameter settings are given as follows. The population size has been set to 100 for each benchmark function, and 50 independent runs have been conducted with 300,000 function evaluations as the termination criterion. The scale parameter has been set to F=0.6 and the crossover rate to CR=0.9. The comparison of the original version of DE (denoted as DE/rand/1/bin) and the new version of DE (denoted as DE/deg/1/bin) described in detail in the previous section and the related experimental results of benchmark set are shown in Table 1, where “Mean Error” and “Std Dev” indicate the average and standard deviation of the function error values obtained in 50 independent runs. To make a fair comparison, the Wilcoxon signed-rank test at the α=0.05 level is conducted between DE/rand/1/bin and DE/deg/1/bin.
Experimental Results of DE/rand/1/bin and DE/deg/1/bin over 50 Independent Runs on 28 Test Functions of 30 Variables with 300,000 Function Evaluations.
| Function | DE/rand/1/bin | DE/deg/1/bin | |||
|---|---|---|---|---|---|
| Mean Error | Std Dev | Mean Error | Std Dev | ||
| Unimodal functions | |||||
| f1 | 3.80E–12 | 5.63E–12 | 1.01E–29 | 2.13E–29 | + |
| f2 | 1.59E+06 | 6.74E+05 | 4.66E+05 | 3.26E+05 | + |
| f3 | 5.29E+04 | 4.14E+04 | 3.41E+03 | 2.34E+04 | + |
| f4 | 1.73E+04 | 4.36E+03 | 2.01E+03 | 8.89E+02 | + |
| f5 | 3.16E–08 | 1.83E–08 | 4.33E–25 | 8.99E–25 | + |
| Multimodal functions | |||||
| f6 | 9.27E+00 | 2.67E+00 | 6.24E+00 | 3.33E+00 | + |
| f7 | 6.92E+00 | 2.51E+00 | 1.78E–01 | 2.25E–01 | + |
| f8 | 2.09E+01 | 5.20E–02 | 2.10E+01 | 6.19E–02 | ≈ |
| f9 | 3.94E+01 | 9.38E–01 | 1.20E+01 | 4.21E+00 | + |
| f10 | 2.17E–03 | 2.59E–03 | 1.01E–02 | 5.33E–03 | – |
| f11 | 1.77E+02 | 1.60E+01 | 1.85E+01 | 5.52E+00 | + |
| f12 | 1.99E+02 | 1.08E+01 | 3.63E+01 | 1.67E+01 | + |
| f13 | 2.00E+02 | 1.26E+01 | 5.88E+01 | 2.70E+01 | + |
| f14 | 6.65E+03 | 4.27E+02 | 7.71E+02 | 2.58E+02 | + |
| f15 | 7.19E+03 | 3.00E+02 | 5.92E+03 | 1.08E+03 | + |
| f16 | 2.47E+00 | 2.47E–01 | 2.36E+00 | 3.54E–01 | + |
| f17 | 2.06E+02 | 1.89E+01 | 5.60E+01 | 5.62E+00 | + |
| f18 | 2.06E+02 | 1.88E+01 | 5.11E+01 | 5.75E+00 | + |
| f19 | 1.68E+01 | 8.98E–01 | 3.05E+00 | 6.95E–01 | + |
| f20 | 1.25E+01 | 2.42E–01 | 1.19E+01 | 4.79E–01 | + |
| Composition functions | |||||
| f21 | 3.39E+02 | 4.90E+01 | 3.37E+02 | 4.85E+01 | + |
| f22 | 6.96E+03 | 3.87E+02 | 7.27E+02 | 2.23E+02 | + |
| f23 | 7.62E+03 | 2.84E+02 | 6.03E+03 | 9.93E+02 | + |
| f24 | 2.04E+02 | 1.07E+00 | 2.00E+02 | 1.56E–01 | + |
| f25 | 1.97E+02 | 2.83E+01 | 2.31E+02 | 2.07E+01 | + |
| f26 | 2.00E+02 | 2.52E–02 | 2.00E+02 | 9.52E–03 | + |
| f27 | 4.40E+02 | 9.10E+01 | 3.05E+02 | 5.89E+00 | + |
| f28 | 3.00E+02 | 5.39E–05 | 3.00E+02 | 0.00E+00 | + |
Notations “+,” “≈,” and “–” indicate that DE/deg/1/bin provided better, similar, or worse results than DE/rand/1/bin.
DE/deg/1/bin provided better results in all five unimodal functions f1–f5 (see Table 1). Better results have also been achieved for the step function f6, noisy quartic function f7, and multimodal functions f9, and f11–f20, and for all composition functions f21–f28. In the case of the multimodal test function f8, the results are comparable. The DE/deg/1/bin provided worse results only in one from 28 test functions from the benchmark set, namely in the case of the multimodal function f10.
4 Conclusion
In this work, the LSN is used to model the dynamics of the DE algorithm. Each SIN is created only by individuals contributing to the improvement of the population and represents one generation of the DE algorithm. On the basis of this model, a new principle of the solution vector selection to generate a mutation vector has been presented. For each generation, one SIN is created and the strength of each node is calculated. In the next generation, the solution vectors are selected on the basis of this strength, which means that individuals with a higher node strength have a higher probability to be selected in the mutation operation to create a mutation vector. In this novel method, a mutation vector can be generated by two or three different solution vectors, or the mutation vector is replaced by one randomly selected solution vector. In this case, parameters of a target vector are just combined with parameters of this one solution vector. The performance of this novel algorithm has been tested on a 28-test function from the well-known benchmark CEC 2013, and the results have been compared with the results provided by the original version of DE. The novel algorithm denoted as DE/deg/1/bin provided better results in most of the test functions, more precisely in 26 from 28 test functions. In one case, the results were comparable and in one case the novel algorithm has provided worse results than the original version of DE.
Acknowledgments
The following grants are acknowledged for the financial support provided for this research: Grant Agency of the Czech Republic – GACR P103/15/06700S, partially supported by Grant of SGS No. SP2015/142, VSB, Technical University of Ostrava.
Bibliography
[1] S. Boccaletti, V. Latorab, Y. Morenod, M. Chavezf and D. U. Hwanga, Complex networks: structure and dynamics, Phys. Rep.424 (2006), 175–308.10.1016/j.physrep.2005.10.009Search in Google Scholar
[2] J. Brest, S. Greiner, B. Boskovic, M. Mernik and V. Zumer, Self-adapting control parameters in differential evolution: a comparative study on numerical benchmark problems, IEEE Trans. Evolut. Comput.10 (2006), 646–657.10.1109/TEVC.2006.872133Search in Google Scholar
[3] N. A. Christakis and J. H. Fowler, The spread of obesity in a large social network over 32 years, N. Eng. J. Med.357 (2007), 370–379.10.1056/NEJMsa066082Search in Google Scholar PubMed
[4] D. Davendra, I. Zelinka, R. Senkerik and M. Pluhacek, Complex network analysis of discrete self-organising migrating algorithm, in: Nostradamus 2014: Prediction, Modeling and Analysis of Complex Systems, pp. 161–174, 2014.10.1007/978-3-319-07401-6_16Search in Google Scholar
[5] J. H. Fowler and N. A. Christakis, Dynamic spread of happiness in a large social network: longitudinal analysis over 20 years in the Framingham Heart Study, BMJ337 (2008), a2338.10.1136/bmj.a2338Search in Google Scholar PubMed PubMed Central
[6] J. J. Liang, B. Y. Qu, P. N. Suganthan and A. G. Hernández-Daz, Problem definitions and evaluation criteria for the CEC 2013 special session on real-parameter optimization, Computational Intelligence Laboratory, Zhengzhou University, Zhengzhou, China and Nanyang Technological University, Singapore, Technical Report 201212, 2013.Search in Google Scholar
[7] R. Mallipeddi, P. N. Suganthan, Q. K. Pan and M. F. Tasgetiren, Differential evolution algorithm with ensemble of parameters and mutation strategies, Appl. Soft Comput.11 (2011), 1679–1696.10.1016/j.asoc.2010.04.024Search in Google Scholar
[8] L. Mercken, C. Steglich, P. Sinclair, J. Holliday and L. Moore, A longitudinal social network analysis of peer influence, peer selection, and smoking behavior among adolescents in British schools, Health Psychol.31 (2012), 450.10.1037/a0026876Search in Google Scholar PubMed
[9] J. Ren, X. Yang, Q. Zhu, L. X. Yang and C. Zhang, A novel computer virus model and its dynamics, Nonlinear Anal. Real World Appl.13 (2012), 376–384.10.1016/j.nonrwa.2011.07.048Search in Google Scholar
[10] R. Storn and K. Price, Differential evolution – a simple and efficient adaptive scheme for global optimization over continuous spaces, in: ICSI, IEEE, 3 1995.Search in Google Scholar
[11] S. Uddin, L. Hossain and R. T. Wigand, New direction in degree centrality measure: towards a time-variant approach, Int. J. Inf. Technol. Decis. Mak.13 (2014), 865–878.10.1142/S0219622014500217Search in Google Scholar
[12] S. Uddin, A. Khan and M. Piraveenan, A set of measures to quantify the dynamicity of longitudinal social networks, Complexity (2015).10.1002/cplx.21690Search in Google Scholar
[13] Y. Wang, Z. Cai and Q. Zhang, Differential evolution with composite trial vector generation strategies and control parameters, Evol. Comput.15 (2011), 55–66.10.1109/TEVC.2010.2087271Search in Google Scholar
[14] Y. C. Wu, W. P. Lee and C. W. Chien, Modified the performance of differential evolution algorithm with dual evolution strategy, in: International Conference on Machine Learning and Computing, 3, pp. 57–63, 2011.Search in Google Scholar
[15] I. Zelinka, D. D. Davendra, R. Senkerik and R. Jasek, Do evolutionary algorithm dynamics create complex network structures?, in: Proceedings of the 12th WSEAS international conference on Communications, 2011.10.25088/ComplexSystems.20.2.127Search in Google Scholar
[16] I. Zelinka, D. D. Davendra, V. Snasel, R. Senkerik and R. Jasek, Preliminary investigation on relations between complex networks and evolutionary algorithms dynamics, in: Computer Information Systems and Industrial Management Applications(CISIM), 2010.10.1109/CISIM.2010.5643674Search in Google Scholar
[17] Y. Zhou, X. Li and L. Gao, A differential evolution algorithm with intersect mutation operator, Appl. Soft Comput.13 (2013), 390–401.10.1016/j.asoc.2012.08.014Search in Google Scholar
©2017 Walter de Gruyter GmbH, Berlin/Boston
This article is distributed under the terms of the Creative Commons Attribution Non-Commercial License, which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
