Real-Time Action Recognition based on Enhanced Motion Vector Temporal Segment Network

EasyChair Preprint 730

Abstract

At present, the method based on two-stream network has achieved good recognition performance in action recognition, however, its real-time performance is obstructed due to the high computational cost of optical flow. Temporal Segment Network (TSN), a successful example based on the two-stream network, achieves high recognition performance but cannot be processed in real time. In this paper, the motion vector TSN (MV-TSN) is proposed by introducing the motion vector into temporal segment networks, which greatly speeds up the processing speed of TSN. In order to solve the problem of performance degradation caused by the motion vectors lacking fine structure information, we propose a knowledge transfer strategy, which initializes the MV-TSN with the fine knowledge learned by optical flow. The experimental results show that the proposed method achieves a comparable recognition performance to the previous state-of-the-art approaches on UCF-101 and HMDB-51, and the processing speed is 206.2 fps, which is 13 times of the original TSN.

Keyphrases: Temporal Segment Network, action recognition, knowledge transfer strategy, motion vector, real-time processing

@booklet{EasyChair:730,
  author    = {Xue Bai and Enqing Chen and Haron Chweya Tinega},
  title     = {Real-Time Action Recognition based on Enhanced Motion Vector Temporal Segment Network},
  howpublished = {EasyChair Preprint 730},
  year      = {EasyChair, 2019}}