Activity-Aware Attributes for Zero-Shot Driver Behavior Recognition

Abstract

In real-world environments, such as the vehicle cabin, we have to deal with novel concepts as they arise. To this end, we introduce ZS-Drive&Act – the first zero-shot activity classification benchmark specifically aimed at recognizing previously unseen driver behaviors. ZS-Drive&Act is unique due to its focus on fine-grained activities and presence of activity-driven attributes, which are automatically derived from a hierarchical annotation scheme. We adopt and evaluate multiple off-the-shelf zero-shot learning methods on our benchmark, showcasing the difficulties of such models when moving to our application-specific task. We further extend the prominent method based on feature generating Wasserstein GANs with a fusion strategy for linking semantic attributes and word vectors representing the behavior labels. Our experiments demonstrate the effectiveness of leveraging both semantic spaces simultaneously, improving the recognition rate by 2.79%.