Performance Comparison of Particle Swarm Optimization , Differential Evolution and Artificial Bee Colony Algorithms for Fuzzy Modelling of Nonlinear Systems

This paper presents the results of the nonlinear system modelling approach based on the use of fuzzy rules optimized by different population based optimization algorithms. Fuzzy rule based models with different number of the rules are used to describe the some nonlinear systems in the literature. Firstly, parameters of the fuzzy models are determined by the artificial bee colony (ABC) algorithm. To demonstrate the efficiency of the ABC algorithm, its modelling ability is compared with the other two powerful population based algorithms, particle swarm optimization (PSO) and differential evolution algorithm (DEA). Simulation results show that a successful model performance with good description ability in the modelling of nonlinear or complex systems can be obtained by using one of the population based algorithms in design of the fuzzy rule based models. DOI: http://dx.doi.org/10.5755/j01.eie.22.5.16336

1 Abstract-This paper presents the results of the nonlinear system modelling approach based on the use of fuzzy rules optimized by different population based optimization algorithms.Fuzzy rule based models with different number of the rules are used to describe the some nonlinear systems in the literature.Firstly, parameters of the fuzzy models are determined by the artificial bee colony (ABC) algorithm.To demonstrate the efficiency of the ABC algorithm, its modelling ability is compared with the other two powerful population based algorithms, particle swarm optimization (PSO) and differential evolution algorithm (DEA).Simulation results show that a successful model performance with good description ability in the modelling of nonlinear or complex systems can be obtained by using one of the population based algorithms in design of the fuzzy rule based models.Index Terms-Artificial bee colony; fuzzy modelling; nonlinear system modelling.

I. INTRODUCTION
An important problem in the design of fuzzy rule based models is that the model parameters need to be optimally determined for input-output data of the systems to be considered.One possible way to overcome this difficulty is to use an artificial intelligence based optimization algorithm which can provide impressive solutions in many engineering problems.The studies presented in literature have shown that structure of the algorithm has an important role to achieve the modelling of the highly nonlinear or complex systems.By using population based heuristic algorithms such as genetic algorithms (GAs), tabu search (TS), simulated annealing (SA) algorithm, differential evolution algorithm (DEA), particle swarm optimization (PSO), artificial bee colony (ABC) algorithm, very effective and accurate model performances can be rapidly found in a systematic way relied on the intelligently search of the solution space.Artificial bee colony (ABC) algorithm presented by Karaboga [1]- [4] is one of these promising optimization methods.
To identify the fuzzy models, many approaches based on the use of evolutionary algorithms have been reported in literature [5]- [24].A standard GA was used by Siarry and Guely to optimize a Takagi-Sugeno type fuzzy rule base [5].To identify the Takagi-Sugeno type fuzzy models for nonlinear systems, another GA based modelling approaches were reported by Wu and Yu [7], and Du and Zhang [8] in the literature.Bagis [9] presented a study based on the use of TS algorithm for the optimum determination of the membership functions of the fuzzy rules that provide the management of the spillway gates in a dam reservoir.The results of the other study based on the use of TS algorithm in the fuzzy rules for nonlinear system modelling was given in [10].For nonlinear system modelling and control, an important study about evolutionary design of fuzzy rules without any assumed rule base structure was presented by Kang et al. [11].A PSO based method to automatically determine the fuzzy rule numbers and membership functions was proposed by Chen [12].Zhao et al. [15] proposed a PSO based method for automatically extracting Takagi-Sugeno fuzzy model from input-output data.In the method proposed by Chen et al. [17], a PSO algorithm with different length of particles is used to design fuzzy rule base automatically.The use of simulated annealing algorithm to optimize the membership functions of Takagi-Sugeno type rules was investigated by Guely et al. [18].To learn the Takagi-Sugeno fuzzy model parameters, Su and Yang [20] proposed a DEA based modelling approach.In the study of Habbi et al. [23], an ABC based approach to obtain the structures and parameters of the Takagi-Sugeno type fuzzy systems was reported.A performance comparison of the Sugeno and Mamdani type fuzzy models optimized by the ABC algorithm for nonlinear system modelling was presented by Bagis and Konar [24].In the study, the performance of the ABC algorithm was tested for different numbers of the fuzzy rules and it was shown that the accuracy of the fuzzy models can be significantly improved by using the ABC algorithm.
This paper presents the results of a fuzzy modelling approach based on the use of ABC, PSO and DEA algorithms for nonlinear system modelling.The performances of the fuzzy models optimized are tested by using different rule numbers for two nonlinear systems in the literature.The simulation results obtained from the algorithms based fuzzy models are compared with each other and with other methods given in the literature.The rest of paper is organized as follows: The next section briefly presents the algorithms used in this study.Definition of the optimized fuzzy rule base structure is given in the following section.Simulation results, comparisons and conclusions are presented in the subsequent sections.

II. THE ALGORITHMS USED IN THE STUDY A. Artificial Bee Colony (ABC) Algorithm
Artificial bee colony (ABC) algorithm presented by Karaboga simulates the intelligent foraging behaviours of honey bee swarm [1]- [4].In this population based algorithm honey bees are categorized by three groups of bees: employed, onlooker, and scout bees.Each solution in the search space is defined by the parameters which specify the position of a food source.Employed bees determine the food sources (possible solutions) and their neighbourhoods (new solutions).On the other hand, onlooker bees evaluate the nectar amounts (qualities of the solutions-fitness value) of the food sources.Thus, by using this information, the selection of the new possible food sources is provided.The qualities of the solutions during the search procedure are improved by using the main control parameters of the algorithm such as colony size, maximum cycles, and limit value (predetermined number of cycles).
In general, a mathematical description that represents the position of the new possible food sources with more quality in the position i.In here, k is the number of food sources randomly determined, and j is the number of the optimization parameters.Thus, V is a new possible solution in the neighbourhood of the old solution X .The parameters of X and X can be defined as the current and neighbour solutions, respectively

B. Particle Swarm Optimization (PSO)
In PSO algorithm based on the use of a population that consists of a set of particles, solution diversity and improvement of the solution quality is achieved by modifying the positions and velocities of the particles [15], [17], [25].For the particles and collective swarm, best values and global best values are saved by the PSO, and in case of necessity they are used.
In obtaining the new position and velocity values for the particles, the definitions given in (2) and (3) are used.In these equations, i is the particle index; v and x are the velocity and position of the particle, respectively; w is the inertia weight; P and G are the best position values obtained by the particle and swarm, respectively; c1 and c2 are the positive constants defined as the cognitive and social parameters; and r1 and r2 are the random values in the interval [0, 1]:

C. Differential Evolution Algorithm (DEA)
In DEA that is a population based heuristic method presented by Storn and Price [26], [27], the differences between the solutions are used for the production of the new possible solutions.The improvement of the solutions is accomplished by using crossover, mutation, and selection operations [20].The mutation operator rather than crossover operation has an effective role in the performance of the algorithm.For new solutions, differences between the vectors defining possible solutions are multiplied by some coefficients called as scaling factor (F).In such an operation for i th solution can be given as the following where F is the scaling factor in the range of [0,1], x1, x2, and x3 are randomly chosen solution vectors.
The control parameters of the algorithms used in this study are set as follows: For the ABC algorithm, colony size = 30, limit value = (colony size/2) × (optimized parameter number); for the PSO method, population size = 30, cognitive and social parameters, c1 = c2 = 2, inertia weight factor, w = (maximum iterationiteration)/(maximum iteration); for the DEA, population size = 30, crossover rate = 0.9, scaling factor(F) = 0.8.Each modelling study was repeated for 30 times at least, and, the best values obtained from these studies are noted in the tables.

III. FUZZY RULE BASE STRUCTURE USED IN THE STUDY
In this study, the fuzzy rule base structure presented by Bagis [10] is employed.According to fuzzy model structure given in Fig. 1, in order to characterize the triangular type membership functions (MFs) of the input variables, three numerical values are used by the fuzzy reasoning mechanism.The output values are defined by singleton values.The parameter matrix representing the membership functions and rules, and a fuzzy rule base with two rules for a sample system that have two inputs and one output are presented in Fig. 1.Apart from the parameters of the input and output membership functions, the parameter matrix has an adjustment parameter (p11, p12 etc.) for each input MFs.Thus, for a system with two inputs and one output, 9 numerical values are used to define a fuzzy rule.
In this study, following reasoning mechanism is used to obtain the final y* value [10] * 1 1 , where r is number of fuzzy rules, i  is a weighted value calculated for ith rule as in where k is input data number,   To evaluate the performance of the fuzzy models, the performance index used in this study is the mean squared error (MSE) as given in the literature.

IV. SIMULATION RESULTS
In this paper, two numerical modelling problems given in the literature are used to investigate the performance of the algorithms for identification of the fuzzy models: an antenna modelling problem and Box-Jenkins gas furnace problem.For this aim, three algorithms are tested by using different number of rules such as 2, 3, 4, 5, and 10, and different maximum cycles (1000, 1500 etc.).This paper presents the best results of these studies for colony size of 30.The results obtained from the ABC based fuzzy models are compared with those obtained DEA and PSO based fuzzy models.In the simulations, the Matlab programming package and Intel Pentium 2800MHz computers are used [28], [29].

A. Antenna Modelling Problem
The essential of this problem given in the literature is to estimate the bandwidth of a rectangular microstrip antenna (MSA) by using a measured data set [24], [30], [31], [32].According to literature, determination of the measured bandwidth (BWme) parameter can be achieved by using three parameters the geometry of a rectangular MSA: the ratio (h/λd) between the thickness of the dielectric substrate (h) and wavelength in the substrate (λd) in the MSA, antenna width (W), and dielectric loss tangent (tanδ).Therefore, the fuzzy rule based models that have three inputs and one output are used in this study.In such a case, the number of optimized parameters is 13 for a rule.In the obtaining of the fuzzy models based on the use of the algorithms, while the training data number is 27, test data number is 6 as used in the literature.Normalization intervals for the variables of inputs (h/λd, W, tanδ) and output (BWme) are selected as [0.005, 0.25], [5,25], [0.0005, 0.0025], and [0.5, 25], respectively.
The minimum MSE values obtained for different fuzzy models with different rule sizes are presented in Table I.From this table, it is shown that the minimum MSE value is obtained from DEA based 10-rules fuzzy model as 0.0044 in maximum cycle of 1000.On the other hand, the MSE values of the ABC and PSO based fuzzy models with 10 rules in 1000 cycle are found as 0.0201, and 0.7604, respectively.The test values of these rule structures in the ABC, DEA, and PSO are 0.6785, 1.1610, and 0.9030, respectively.Similarly, after 30 runs of the ABC, DEA, and PSO approaches, the standard deviation values are noted as 0.0309, 0.3658, and 3.7544, respectively again.These results point out the effectiveness of the ABC and DEA algorithms to optimize the parameters of the fuzzy models for a hard nonlinear modelling problem.This fact is clearly seen from Fig. 2 that presents the outputs of the ABC, DEA, and PSO based fuzzy models with 10 rules and the original (measured) bandwidth values.For optimized fuzzy models, the variation of iteration (or cycle)-MSE during the first 250 iterations of 1000 iterations are graphically given in Fig. 3.And a comparison of the ABC, DEA, and PSO based fuzzy models and the other approaches in the literature are presented in Table II.As shown in Fig. 3, for the first 100 iteration, the MSE performances of the DEA and PSO based fuzzy models with 10 rules are better than those obtained by the ABC based model.On the other hand, PSO based model does not exhibit a significant improvement after the first 25 iterations.In using the ABC algorithm, the improvement in the model performance is slow but continual during the cycles.When the all of the values in Table I

B. Box-Jenkins Gas Furnace Problem
In this subsection, Box-Jenkins gas furnace problem that consists of a data set including input-output measurements of 296 pairs is used to obtain a fuzzy rule based model [10], [11], [24], [33]- [43].The process has a single input u(t) (gas flow rate) and a single output y(t) (CO2 concentration).In this study, the inputs of the optimized fuzzy models are used as u(t-4), and y(t-1), and the output of the models is accepted as y(t).The normalization intervals of the inputs and output are selected as [-3, 3], [44,62], and [44, 62], respectively.The results including the minimum MSE values for the optimized fuzzy models are given in Table III.As shown from this table, minimum MSE values in the use of 10 rules and 1500 cycles for the ABC, DEA, and PSO approaches are found as 0.1385, 0.1278, and 0.1542, respectively.The standard deviation values of these algorithms for 10 rules are noted as 0.0059, 0.0090, and 0.0295, respectively again.The outputs of the fuzzy models with 10 rules and the original output of the system are given comparatively in Fig. 4. Furthermore, iteration-MSE variations of the fuzzy models are given in Fig. 5.For clarity, the first 150 iterations of 1500 iterations are indicated in this figure .The MSE values of our study and the other approaches in the literature are presented in Table IV.A good agreement between the fuzzy model outputs and the original output can be clearly seen from Fig. 4. Figure 5 exhibits the remarkable reductions in the MSE values that provided by the algorithms during the first 50 iterations.When considering the performances of the PSO and DEA methods during the first 25 iterations, we can say that the convergence speed of the ABC algorithm is a little slower than the other algorithms for this problem.However, the results in the Table III and Table IV prove the undeniable competitiveness of the ABC algorithm according to other methods.

V. CONCLUSIONS
This paper presents a useful investigation about the effectiveness of the population based three popular algorithms in the fuzzy modelling of the nonlinear systems, and it compares the performances of these algorithms, namely, ABC, DEA and PSO.For this aim, the fuzzy rule based models with different rule number optimized by the algorithms is applied to the popular nonlinear systems given in the literature.Simulation results show that the ability to define the nonlinear or complex systems of the fuzzy rule based models can be significantly improved by using the DEA and ABC algorithms especially, and the competitive solutions for difficult engineering problems can be produced.Moreover, these results encourage that, after some additional improvements, the computational capabilities of the algorithms can be increased to perform the more desired modelling performance.

Fig. 1 .
Parameter matrix for defining the membership functions and rules(a); A sample fuzzy rule base with 2 rules (b).

Fig. 4 .
Fig. 4. The outputs of the original and fuzzy models with 10 rules for Box-Jenkins problem (•• PSO model, --ABC model,  DEA model).

TABLE I .
MINIMUM MSE VALUES FOR DIFFERENT FUZZY RULE BASED MODELS OBTAINED BY USING ALGORITHMS.

TABLE II .
COMPARISON OF OUR STUDY AND THE OTHER METHODS IN THE LITERATURE FOR ANTENNA PROBLEM.

TABLE III .
MINIMUM MSE VALUES OBTAINED BY DIFFERENT FUZZY MODELS FOR BOX-JENKINS PROBLEM.

TABLE IV .
COMPARISON OF OUR STUDY AND THE OTHER METHODS IN THE LITERATURE FOR BOX-JENKINS PROBLEM.