REAL-TIME FEEDBACK MECHANISM BASED ON IMAGE ANALYSIS IN MOTOR FUNCTION RECOVERY

Abstract

In the motor function rehabilitation of patients, image analysis technology is being widely used in the rehabilitation treatment of patients. Combined with images, feedback on the patients’ rehabilitation status can be provided, and the treatment plan can be adjusted in a timely manner. Existing image analysis technologies mostly use convolutional neural networks (CNN) or long short-term memory (LSTM) network to analyze continuous motion movements. They have good enough performance in terms of accuracy but may lack real-time performance. Based on this, this paper proposes a CNN-TCN architecture that combines the ResNet-50 model of the CNN structure and the temporal convolutional networks (TCN), a variant of the recurrent neural networks (RNN), and uses headphones, videos, etc., for real-time feedback. To verify the effect of the architecture and realize the real-time feedback mechanism of motion function based on image analysis, this experiment selects HMDB51 and KTH datasets as the initial datasets for training, supplemented with common daily action data such as walking, bending, and arm swinging, etc. Then 50 patients who need rehabilitation are recruited as volunteers to verify the results. The results are evaluated using three indicators: accuracy, recall, and feedback time. It is found that the accuracy and recall of CNN alone are 76% and 74%, and the accuracy and recall of LSTM alone are 83% and 84%, while the accuracy and recall of CNN-TCN are 86% and 87%. The feedback time of CNN, LSTM, and CNN-TCN is basically 340 to 400 milliseconds, 360 to 430 milliseconds, and 290-360 milliseconds respectively. CNN-TCN is better than CNN and LSTM in accuracy and also outperforms CNN and LSTM in inference time. Therefore, CNN-TCN is a better choice while ensuring high accuracy and good effectiveness.