Posture and Appearance Fusion Network for Driver Distraction Recognition

EasyChair Preprint 9212

Abstract

Distracted driving is the act of driving while engaged in other activities, such as using a cell phone, texting, eating, or reading, which takes the driver' attention away from the road. Nowadays, the distracted driving detection models based on deep learning can extract critical information from video data to characterize the driving behavior process. But the distraction driving method based solely on appearance features cannot essentially eliminate the noise impact of the complex environment on the model, and the distracted driving recognition method based solely on skeletal information is unable to recognize the joint action of the human body and the objects. Therefore,the development of an accurate distracted driving detection model has become challenging. In this paper, we propose a distracted driving recognition model MFD-former based on the fusion of posture and appearance. First, a feature extraction module is proposed to extract skeleton data(i.e., posture) and appearance features(i.e., descriptors), which are merged by a graph neural network. Then, the two kinds of information are input into the MFD-former encoder module, and the self-attention mechanism quickly extracts the sparse data. Finally, the classification results of distracted driving are obtained by extracting the classification labels through the MLP Head. The MFD-former model outperforms existing models. It achieved 95.1% accuracy on the State Farm dataset and 90.24% accuracy on the self-built Train Drivers dataset.

Keyphrases: Attention Mechanism, Driver distraction recognition, Graph Neural Network, Heterogeneous Information Fusion

@booklet{EasyChair:9212,
  author    = {Hao Yu and Chong Zhao and Xing Wei and Yan Zhai and Zhen Chen and Guangling Sun and Yang Lu},
  title     = {Posture and Appearance Fusion Network for Driver Distraction Recognition},
  howpublished = {EasyChair Preprint 9212},
  year      = {EasyChair, 2022}}