CaRS: A Causal Intervention Segmentation Framework and Benchmark Dataset for Autonomous Driving under Transitional Weather Conditions

Autonomous vehicles must excel in safety-critical perception tasks, especially in adverse weather conditions. In addition, transitional weather shifts in nature, such as sunny to rainy, rainy to cloudy, etc., pose abrupt illumination changes that can distort object boundaries and degrade segmentation performance. Existing research focuses mainly on segmentation in clear and discrete weather conditions, leaving a gap in addressing the issues of transitional weather scenarios. Hence, we propose a novel method called causal road and rest segmentation (CaRS) that utilizes causal intervention to mitigate the confounding bias due to transitional weather changes. We use dual complementary attention modules, one for causal and another for confounding feature extraction. These modules complement each other and are fine-tuned via an adversarial min-max approach to reduce confounding bias and enhance segmentation performance. Also, our CaRS method concurrently performs road semantic segmentation and instance segmentation of vehicles and pedestrians. Further, we introduce a transitional weather-driving dataset for segmentation (TWDS16) using a spurious correlation generator that leverages data interpolation to produce 16 weather transitions. We evaluate the performance of CaRS on TWDS16, along with three other benchmark datasets, namely, Foggy Cityscapes, RainCityscapes, and BDD100K. The experimental results validate the efficacy of the proposed method in mitigating confounding influences, leading to improved \textit{mIoU} for semantic segmentation and \textit{mAP} for instance segmentation across diverse datasets