Color Preserving CMOS-SPAD Fusion for Multi-Frame HDR

High dynamic range (HDR) imaging aims to simultaneously capture a large range of illuminance levels, often observed in real world scenes. An HDR image is conventionally captured on a CMOS sensor by combining multiple frames with different exposure values. Ultimately, the dynamic range is limited by the read noise of the CMOS at low-light and by the full-well capacity at high-light. Single-photon avalanche diode (SPAD) imagers display high HDR capabilities due to sensitivity from individual photons up to high photon fluxes. In this work, we demonstrate that fusing multi-frame raw-CMOS RGB and monochrome SPAD images can enhance the dynamic range and color accuracy in real-world settings. To fully leverage the SPAD data, we propose a Linearisation-Upsample processing block that linearises the intrinsically nonlinear response of SPAD-QIS and accounts for the low spatial resolution of current commercial SPAD technology. We demonstrate a clear advantage by including a SPAD image into a multiframe HDR pipeline with comprehensive evaluation on HDR datasets and real-world data. We also address the lack of public SPAD datasets by providing two raw CMOS-SPAD datasets for multi-frame HDR. Both datasets are available for download here: https://tinyurl.com/rawcmosspadhdr