Video analysis technology has always been an essential branch of computer vision. Analyzing the movie genres is beneficial to pushing relevant and exciting content to target customer groups to achieve precision marketing. There are some researches on movie trailers to classify movie genres. However, most of them are based on movies' auditory and visual content using various machine learning models or neural network models for classification. This paper considers the features learned using scene-based neural network models in movie genre classification. This paper proposes a hybrid PlacesNet-LSTM (long short-term memory) model for movie trailer genre classification. To compare the performance, the paper also studies two schemes using various video and audio features based on multiple machine learning models and LSTM, respectively. The experimental results show that the PlacesNet-LSTM model on scene recognition achieves the best classification performance in various combinations.