Comparing Imperialist Competitive Algorithm with Backpropagation Algorithms for Training Feedforward Neural Network

Volume 14, Issue 3, pp 193-204 http://dx.doi.org/10.22436/jmcs.014.03.02

Download PDF

Download XML

• Export citations
  • CITATIONS & REFERENCES
  • EndNote (.ENW)
  • Mendeley, JabRef (.BIB)
  • Papers, Zotero, Reference Manager, RefWorks (.RIS)
  • ARTICLE CITATION
  • EndNote (.ENW)
  • Mendeley, JabRef (.BIB)
  • Papers, Zotero, Reference Manager, RefWorks (.RIS)
  • REFERENCES
  • EndNote (.ENW)
  • Mendeley, JabRef (.BIB)
  • Papers, Zotero, Reference Manager, RefWorks (.RIS)

• 3285 Downloads
• 4599 Views

Authors

Maryam Zanganeh - Department of computer engineering, Central Tehran Branch, Islamic Azad University, Tehran, Iran. Seyed Javad Mirabedini - Department of computer engineering, Central Tehran Branch, Islamic Azad University, Tehran, Iran.

Abstract

Artificial Neural Networks (ANN) and evolutionary algorithms are two relatively young research areas that were subject to a steadily growing interest during the past years. The use of ANN has been proved to be a cost-effective technique. It is very important to choose a suitable algorithm for training a neural network. Mostly Back Propagation (BP) algorithm is a gradient descent algorithm (a first-order optimization algorithm) on the error space, which most likely gets trapped into a local minimum and has very slow convergence. This shortcoming can be removed by global searching ability of the evolutionary algorithms such as Imperialist Competitive Algorithm (ICA) which is a new evolutionary algorithm based on the human's socio-political evolution. This investigation provides a comparison between training a neural network with BP algorithms used for training Feed-forward Neural Networks (FNN) and ICA. Among the BP algorithms, Gradient descent, Levenberg–Marquardt, Conjugate gradient descent, Resilient, BFGS Quasi-newton, and One-step secant algorithm are tested then the obtained results will be compared with the results of training the neural network with ICA. Also, Accuracy and Mean Squared Error (MSE) are the main measures selected to assess both models. Also the MSE was used as a criterion to specify optimum number of neurons in the hidden layer. The results showed that ICA approach outperforms the BP for training neural network models.

Share and Cite

Keywords

• Imperialist Competitive Algorithm
• Backpropagation
• Artificial Neural Network.

MSC

•  68T05
•  92B20

References

• [1] S. M. Aquil Burney, Tahseen A. Jilani, time series forecasting using artificial neural network methods for karachi stock exchange, A project at department of computer science, university of Karachi (2002)
  • Google Scholar


• [2] Y. Shang, B.W. Wah, Global Optimization for Neural Networks Training, IEEE Computer, 29 (1996), 45-54.
  • View Article
  • Google Scholar


• [3] A. Abraham, Meta learning evolutionary artificial neural networks, Neuro Computing. , 56 (2004), 1-38.
  • View Article
  • Google Scholar


• [4] Jiancheng Fang, Zhongyu Wang, The combination of immune evolution and neural network for nonlinear time series forecasting Sixth International Symposium on Instrumentation and Control Technology: Sensors, Automatic Measurement, Control and Computer Simulation, Beijing, China (2006)
  • Google Scholar


• [5] Haibin Duan, Linzhi Huang, Imperialist competitive algorithm optimized artificial neural networks for UCAV global path planning, Neurocomputing, (2014)
  • View Article
  • Google Scholar


• [6] A. Iftikhar, M. A. Ansari, S. Mohsin, Performance Comparison between Back propagation Algorithms Applied to Intrusion Detection in Computer Network Systems, 9th WSEAS International Conference on Neural Networks., (2008), 231-236.
  • Google Scholar


• [7] R. Noori, M. S. Sabahi, A. R. Karbassi, Evaluation of PCA and Gamma test techniques on ANN operation for weekly solid waste predicting, J. Environ. Manage. , 91 (2010), 767-771.
  • View Article
  • Google Scholar


• [8] Mohammad Ali Ahmadi, Mohammad Ebadi, Amin Shokrollahi, Seyed Mohammad Javad Majidi., Evolving artificial neural network and imperialist competitive algorithm for prediction oil flow rate of the reservoir, Applied Soft Computin., 13 (2013), 1085-1095.
  • View Article
  • Google Scholar


• [9] Shahram Mollaiy Berneti, Mehdi Shahbazian, an imperialist competitive algorithm-artificial neural network method to predict oil flow rate of the wells, International Journal of Computer Applications. , 26 (2011), 975-8887.
  • Google Scholar


• [10] S. M. Hosseini, A. Al Khaed., A survey on the Imperialist Competitive Algorithm metaheuristic, Implementation in engineering domain and directions for future research., 24 (2014), 1078-1094.
  • View Article
  • Google Scholar


• [11] S. Mirjalili, A. Safa Sadiq, Magnetic optimization algorithm for training multi-layer perceptron, in: IEEE International Conference on Industrial and Intelligent Information, Indonesia., 2 (2011), 42-46.
  • View Article
  • Google Scholar


• [12] M. C. P. de Souto, A. Yamazaki, T. B. Ludernir , Optimization of neural network weights and architecture for odor recognition using simulated annealing, Proceedings of the 2002 International Joint Conference on Neural Networks. , 1 (2002), 547–552.
  • View Article
  • Google Scholar


• [13] K. Hornick, M. Stinchcombe, H. White, Multilayer feed forward networks are universal approximators, Neural Networks , 2 (1989), 359–366.
  • View Article
  • Google Scholar


• [14] N. García-Pedrajas, C. Hervás-Martínez, J. Munoz-Perez, A cooperative co-evolutionary model for evolving artificial neural networks, IEEE Transactions on Neural Networks. , 14 (2003), 575–596.
  • View Article
  • Google Scholar


• [15] T. Ganesan, I. Elamvazuthi, P. Vasant, Solving engineering optimization problems with the Karush–Kuhn–Tucker Hopfield neural networks, International Review of Mechanical Engineering., 5 (2011), 1333-1339.
  • Google Scholar


• [16] S. Verma, B. D. Huey, D. J. Burgess, Scanning probe microscopy method for nanosuspension stabilizer selection, Langmuir, 25 (2009), 12481-12487.
  • Google Scholar


• [17] K. I. Funahashi, On the Approximate Realization of Continuous Mappings by Neural Networks, Neural Networks., 2 (1989), 183-192.
  • View Article
  • Google Scholar


• [18] K. Hornik, Approximation Capabilities of Multilayer Feedforward Networks, Neural Networks. , 4 (1991), 251-257.
  • View Article
  • Google Scholar


• [19] G. Cybenko, Approximation by Super positions of Sigmoidal Function, Math. Contr. Signals Syst. , 2 (1989), 303-314.
  • View Article
  • Google Scholar


• [20] K. Levenberg, A Method for the Solution of Certain Nonlinear Problems in Least Squares, Quarterly of Applied Mathematics. , 2 (1944), 164-168.
  • Google Scholar


• [21] M. S. Mirtalaei, M. Saberi, O. K. Hussain, B. Ashjari, F. K. Hussain, A trust-based bio-inspired approach for credit lending decisions, Computing, 94 (2012), 541-577.
  • View Article
  • Google Scholar


• [22] Xinying Miao, Jinkui Chu, Linghan Zhang, Jing Qiao., An Evolutionary Neural Network Approach to Simple Prediction of Dam Deformation, Journal of Information & Computational Science. , 10 (2013), 1315–1324.
  • Google Scholar


• [23] Lian Kunlei, Zhang Chaoyong, Gaoa Liang, Single row facility layout problem using an imperialist competitive algorithm, Proceedings of the \(41^{st}\) International Conference on Computers & Industrial Engineering. , (),
  • Google Scholar


• [24] E. Atashpaz-Gargari, C. Lucas, Imperialist competitive algorithm: an algorithm for optimization inspired by imperialistic competition, IEEE Congress on Evolutionary Computation., (2007), 4661-4667.
  • View Article
  • Google Scholar