The Wisdom of Crowds: Temporal Progressive Attention
for Early Action Prediction

CVPR 2023

    (*work carried out while at the University of Bristol)

Abstract

Early action prediction deals with inferring the ongoing action from partially-observed videos, typically at the outset of the video. We propose a bottleneck-based attention model that captures the evolution of the action, through progressive sampling over fine-to-coarse scales. Our proposed Temporal Progressive (TemPr) model is composed of multiple attention towers, one for each scale. The predicted action label is based on the collective agreement considering confidences of these towers. Extensive experiments over four video datasets showcase state-of-the-art performance on the task of Early Action Prediction across a range of encoder architectures. We demonstrate the effectiveness and consistency of TemPr through detailed ablations.

Video

BIBTEX

@inproceedings{stergiou2023wisdom,
title={The Wisdom of Crowds: Temporal Progressive Attention for Early Action Prediction},
author={Stergiou, Alexandros and Damen, Dima},
booktitle={IEEE/CVF Computer Vision and Pattern Recognition (CVPR)},
year={2023}
}

Acknowledgements

Funded by the United Nation’s End Violence Fund (iCOP 2.0) and EPSRC UMPIRE (EP/T004991/1).