A Concrete Threshold-Feedback Network for Real-Time On-Sensor Edge Extraction in CMOS Image Sensors

This article proposes a concrete thresholdfeedback network (CTFN) for a prototype column-parallel single-slope ADC (CP SS-ADC) CMOS image sensor (CIS) that generates a normal image and a real-time edge image concurrently. The optimal edge threshold varies significantly with scene content and illumination, but conventional global or histogram-based methods (e.g., Otsu) rely on fixed rules and require per-frame computation, resulting in degraded edge quality and higher latency. To address this limitation, CTFN learns a scene-adaptive threshold directly from the current frame and programs it into the on-sensor register with a single write per frame, preserving the existing CP SS-ADC readout pipeline. To enable end-to-end optimization, the discrete threshold selection is relaxed via the concrete (GumbelSoftmax) distribution, allowing gradients to propagate through the threshold decision. Training is guided by a joint objective that combines SSIM-based edge fidelity and classification loss, enabling the model to optimize threshold selection and downstream recognition simultaneously. Experiments on ImageNet classification with four edge operators (Sobel, Prewitt, Scharr, and Laplacian) show that CTFN improves Top-1 accuracy by 0.21.1 percentage points over Global and Otsu methods while reducing inference latency to 0.526 ms per frame-corresponding to a 50.28%–62.59% reduction relative to Global and an 18.83%–40.16% reduction relative to Otsu. Qualitative results further demonstrate that the learned thresholds produce more illumination-robust and structurally consistent edge images. These results indicate that CTFN provides an efficient and practical solution for real-time, scene-adaptive edge extraction in resourceconstrained sensor systems, offering a promising direction for integrating lightweight neural controllers with on-sensor edge processing.