Video identification and analysis is one of the important research contents in the field of computer vision. Among them, the fine-grained action recognition of the video is a more refined and challenging recognition task. The main challenges are the few available fine-grained action recognition datasets that limit the progress of research in this field; fine-grained action-recognition is designed to distinguish subclasses in a large action classification, which are more subtle, usually only by small local differences. Existing fine-grained recognition tasks generally use target detection, attention mechanism and other related methods to find and use the local regional information in the image. However, most of these methods are used for image recognition tasks, so they lack the utilization of timing information for video. This paper uses dual-based method to study fine-grained video action recognition. A fine-grained football video dataset, Football is presented. It consists of live videos of multiple football matches. Initially, we categorized three broad movement types: dribbling, passing, and shooting. Subsequently, these were broken down into a more detailed set of 26 specific movements. All of the experiments presented in this paper will be implemented on this dataset. All methods in this paper will complete related experiments on the Football football dataset and the MPII cooking dataset. In the process of various network optimization, these methods achieve improved results and outperformed the current mainstream methods, which verifies the effectiveness of our methods.