Comparison of Particle Swarm Optimization and Backpropagation Algorithms for Training Feedforward Neural Network
- Department of computer engineering, Tehran Science and Research Branch, Islamic Azad University, Damavand, Iran.
Seyed Javad Mirabedini
- Department of computer engineering, Central Tehran Branch, Islamic Azad University, Tehran, Iran.
An interesting tool for non-linear multivariable modeling is the Artificial Neural Network (ANN) which has been developed recently. The use of ANN has been proved to be a cost-effective technique. It is very important to choose a suitable algorithm for training a neural network. Generally Backpropagation (BP) algorithm is used to train the neural network. While these algorithms prove to be very effective and robust in training many types of network structures, they suffer from certain disadvantages such as easy entrapment in a local minimum and very slow convergence. In this paper, to improve the performance of ANN, the adjustment of network weights using Particle Swarm Optimization (PSO) was proposed as a mechanism and the results obtained were compared with various BP algorithms such as Levenberg-Marquardt and gradient descent algorithms. Each of these networks runs and trains for different learning ratios, activation functions and numbers of neurons within their hidden layer. Among different criteria Mean Square Error (MSE) and Accuracy are the main selected criteria used for evaluating both models. Also the MSE was used as a criterion to specify optimum number of neurons in hidden layer. The results showed that PSO approach outperforms the BP for training neural network models.
Share and Cite
Nasser Mohammadi, Seyed Javad Mirabedini, Comparison of Particle Swarm Optimization and Backpropagation Algorithms for Training Feedforward Neural Network, Journal of Mathematics and Computer Science, 12 (2014), no. 2, 113-123
Mohammadi Nasser, Mirabedini Seyed Javad, Comparison of Particle Swarm Optimization and Backpropagation Algorithms for Training Feedforward Neural Network. J Math Comput SCI-JM. (2014); 12(2):113-123
Mohammadi, Nasser, Mirabedini, Seyed Javad. "Comparison of Particle Swarm Optimization and Backpropagation Algorithms for Training Feedforward Neural Network." Journal of Mathematics and Computer Science, 12, no. 2 (2014): 113-123
- Particle Swarm Optimization
- Artificial Neural Network.
Y. Shang, B. W. Wah , Global Optimization for Neural Networks Training , IEEE Computer. , 29 (1996), 45-54.
A. Abraham , Meta learning evolutionary artificial neural networks, Neuro Computing, 56 (2004), 1-38.
M. S. Mirtalaei, M. Saberi, O. K. Hussain, B. Ashjari, F. K. Hussain, A trust-based bio-inspired approach for credit lending decisions, Computing, 94 (2012), 541-577.
Y. Y. Hsu, C. C. Yang, Fast voltage estimation using ANN , Electric Power System Research, 27 (1993), 1-9.
T. Jain, L. Srivastava, S. N. Singh, Fast voltage contingency screening using Radial Basis Function neural network, IEEE Transactions on Power Systems, 18 (2003), 1359-1366.
R. Govindaraju, A. Rao, Artificial Neural Networks in Hydrology, Kluwer Academic Publishers, Dordrecht (2000)
S. Y. Liong, W. H. Lim, G. N. Paudyal , River stage forecasting in Bangladesh: neural network approach, Journal of Computing in Civil Engineering, 14 (2000), 1-8.
K. W. Chau, C. T. Cheng , Real-time prediction of water stage with artificial neural network approach, Lecture Notes in Artificial Intelligence, 2557 (2002), 715-715.
S. Lahmiri, A comparative study of backpropagation algorithms in financial prediction, International Journal of Computer Science, Engineering and Applications, 1 (2011), 15-21.
F. Hooshyaripor, A. Tahershamsi , Comparing the performance of Neural Networks for Predicting Peak Outflow from Breached Embankments when Back Propagation Algorithms Meet Evolutionary Algorithms, International Journal of Hydraulic Engineering, 1 (2012), 55-67.
S. A. Mirjalili, S. Z. M. Hashim, H. M. Sardroudi, Training feedforward neural networks using hybrid particle swarm optimization and gravitational search algorithm, Applied Mathematics and Computation, 218 (2012), 11125-11137.
M. Yaghini, M. M. Khoshraftar, M. Fallahi, A hybrid algorithm for artificial neural network training, Engineering Applications of Artificial Intelligence, 26 (2013), 293-301.
G. Das, P. K. Patnaik, S. K. Padhy, Artificial Neural Network trained by Particle Swarm Optimization for non-linear channel equalization, Expert Systems with Applications, 41 (2013), 3491-3496.
C. M. Bishop, Neural networks for pattern recognition, Oxford University Press, New York (1995. )
D. B. Fogel , An introduction to simulated evolutionary optimization, IEEE Transactions on Neural Networks, 5 (1994), 3-14.
J. S. R. Jang, C. T. Sun, E. Mizutani , Neuro-Fuzzy and Soft Computing: A Computational Approach to Learning and Machine Intelligence, Prentice-Hall , Upper Saddle River (1997)
M. T. Hagan, M. Menhaj , Training feedforward networks with the Marquardt algorithm, IEEE Trans Neural Networks, 5 (1994), 989-993.
M. Y. El-Bakyr, Feed forward neural networks modeling for K–P interactions, Chaos, Solitons and fractals, 18 (2003), 995-1000.
J. Kennedy, R. Eberhart, Particle swarm optimization, IEEE international conference on neural networks, 4 (1995), 1942-1948.
R. Kiran, S. R. Jetti, G. K. Venayagamoorthy, Online training of generalized neuron with particle swarm optimization, IEEE International Joint Conference on Neural Networks, (2006), 5088-5095.
N. Kwok, D. Liu, K. Tan, An empirical study on the setting of control coefficient in particle swarm optimization, Proceedings of IEEE Congress on Evolutionary Computation, (2006), 823-830.
T. J. Richer, T. M. Blackwell, When is a swarm necessary?, IEEE Congress on Evolutionary Computation, (2006), 1469-1476.
H. A. Abbass, R. Sarker, C. Newton, PDE: A Pareto-frontier Differential Evolution Approach for Multi-objective Optimization Problems, IEEE Congress on Evolutionary Computation, 2 (2001), 971-978.
R. C. Eberhart, Y. Shi , Comparison between genetic algorithms and Particle Swarm Optimization, International Conference on Evolutionary Programming VII, 1447 (1998), 611-616.