Skip to yearly menu bar Skip to main content


Poster

Binarized Semantic Mamba-Transformer for Lightweight Quad Bayer HybridEVS Demosaicing

Shiyang Zhou · Haijin Zeng · Yunfan Lu · Tong Shao · Ke Tang · Yongyong Chen · Jie Liu · Jingyong Su


Abstract:

Quad Bayer demosaicing is the central challenge for enabling the widespread application of Hybrid Event-based Vision Sensors (HybridEVS). Although existing learning-based methods that leverage long-range dependency modeling have achieved promising results, their complexity severely limits deployment on mobile devices for real-world applications. To address these limitations, we propose a lightweight Mamba-based binary neural network designed for efficient and high-performing demosaicing of HybridEVS RAW images. First, to effectively capture both global and local dependencies, we introduce a hybrid Binarized Mamba-Transformer architecture that combines the strengths of the Mamba and Swin Transformer architectures. Next, to significantly reduce computational complexity, we propose a binarized semantic Mamba (BiS-Mamba), which binarizes all projections while retaining the core Selective Scan in full precision. BiS-Mamba also incorporates additional semantic information to enhance global context and mitigate precision loss. We conduct quantitative and qualitative experiments to demonstrate the effectiveness of BMTNet in both performance and computational efficiency, providing a lightweight demosaicing solution suited for real-world edge devices. Codes and models are available in the supplementary materials.

Live content is unavailable. Log in and register to view live content