Workzone3D: A Multimodal Dataset for 3D Work Zone Perception in Autonomous Driving

Work zones are essential to maintain, repair and upgrade our roadways. However, they introduce complex, dynamic and challenging environments for autonomous vehicles to navigate safely. To help address this challenge, we introduce the first publicly available, large-scale, multimodal work zone dataset collected with an autonomous vehicle consisting of multiple synchronized lidars and high-resolution cameras. The dataset covers various work zone elements like cones, barrels and other channelizers. Our dataset, referred to as WorkZone3D, consists of 3D annotation boxes for these objects. We also propose a detailed auto-annotation pipeline that can produce high-quality 3D labels, even for rare classes which do not have pre-trained 3D object detection models to start with. We evaluate a camera+lidar-based deep learning model on this dataset, highlighting the critical role of sensor fusion in accurate 3D localization of such small objects at a distance, often having very few lidar points on them. Our results demonstrate the usefulness of our dataset for generalization to real-world scenarios. We will release both our code and data.