An emerging and interesting research field which focuses on the enhancement of developing an automatic emotion recognition system is Facial Expression Recognition. This technology creates a major impact on various applications such as safety, healthcare, gaming and education. Researchers in this field are working to develop methods that can extract and interpret facial expressions. Deep learning plays a vital role in the field of computer vision technology and Convolutional Neural Network (CNN) is the most significant part in the deep learning. There are few drawbacks in the training algorithm, Stochastic Gradient Descent (SGD) used in CNN. This proposed work overcomes the drawbacks of SGD by introducing an alternative algorithm based on Genetic Algorithm and Particle Swarm Optimization to improve the performance of CNN. This paper presents the novel PSO-GA algorithm and addresses the benefits of PSO-GA algorithm which challenges and outperforms other existing methods and achieves robust and state-of-the art results in various challenging datasets. This hybrid PSO-GA algorithm makes the CNN to identify the facial expressions and emotions very effectively when compared to other methodology.