UNCERTAINTY-GUIDED LIGHTWEIGHT TRANSFORMER FOR JOINT IMAGE DENOISING AND EDGE-PRESERVED ENHANCEMENT IN RESOURCE-CONSTRAINED VISION SYSTEMS

Authors:

Abhishek Singh Rathour,Mohit Kumar Srivastava,Keshav Kumar,Vivek Kumar,Gaurav Sahu,

DOI NO:

https://doi.org/10.26782/jmcms.2026.11.00004

Keywords:

Digital Image Processing,Image Denoising,Low-Light Enhancement,Lightweight Transformer,Uncertainty Estimation,Edge Preservation,Signal Processing,Image Restoration,Resource-Constrained Vision,IoT Imaging.,

Abstract

Background: This work addresses a realistic digital signal processing problem in which low-light acquisition, spatially variant sensor noise, and high-frequency edge attenuation occur simultaneously. Classical denoisers provide useful signal priors but are limited by fixed degradation assumptions, whereas large transformer restorers improve perceptual quality at the cost of high memory use and computation. Objective: This paper proposes an uncertainty-guided lightweight transformer, instead of a conventional heavyweight transformer, for joint image denoising and edge-preserved enhancement. Methods: The proposed UGLT-EPE framework combines a signal-aware preprocessing stem, local convolutional filtering, windowed self-attention, uncertainty-guided fusion, and Laplacian edge consistency. The uncertainty map is defined as a normalized prediction of the local restoration error and is trained against that error with a dedicated calibration loss; it re-weights attention and feature fusion towards unreliable regions while protecting reliable edge structures from over-smoothing. A compact mathematical formulation is developed for the forward degradation model, uncertainty estimation, token gating, edge-aware reconstruction, and the total optimization objective. Simulation results compare the proposed method with BM3D, DnCNN, FFDNet, MIRNet, MPRNet, SwinIR, Restormer, and a NAFNet-style baseline. Results: The proposed method achieves higher PSNR and SSIM under increasing noise levels, a higher edge-preservation F1-score, lower perceptual error, and an improved speed-accuracy balance. Conclusions: The results indicate that signal-guided lightweight attention can provide a strong compromise between restoration accuracy and deployability for edge cameras, UAV vision systems, and low-power IoT imaging nodes.

Refference:

I. A. Dosovitskiy, et al. : ‘An Image Is Worth 16×16 Words: Transformers for Image Recognition at Scale’. Proceedings of the International Conference on Learning Representations (ICLR). 2021. https://arxiv.org/abs/2010.11929
II. A. Foi, M. Trimeche, V. Katkovnik, K. Egiazarian. : ‘Practical Poissonian–Gaussian Noise Modeling and Fitting for Single-Image Raw-Data’. IEEE Transactions on Image Processing. Vol. 17(10), pp: 1737-1754, 2008. 10.1109/TIP.2008.2001399
III. A. Ignatov, R. Timofte, L. Van Gool. : ‘Replacing Mobile Camera ISP with a Single Deep Learning Model’. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 2020. https://arxiv.org/abs/2002.05509
IV. A. Kendall, Y. Gal. : ‘What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?’. Advances in Neural Information Processing Systems (NeurIPS). Vol. 30, pp: 5574-5584, 2017. https://arxiv.org/abs/1703.04977
V. C. Chen, Q. Chen, J. Xu, V. Koltun. : ‘Learning to See in the Dark’. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp: 3291-3300, 2018. https://arxiv.org/abs/1805.01934
VI. C. Guo, C. Li, J. Guo, C. C. Loy, J. Hou, S. Kwong, R. Cong. : ‘Zero-Reference Deep Curve Estimation for Low-Light Image Enhancement’. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp: 1780-1789, 2020. 10.1109/CVPR42600.2020.00185
VII. C. Guo, G. Pleiss, Y. Sun, K. Q. Weinberger. : ‘On Calibration of Modern Neural Networks’. Proceedings of the 34th International Conference on Machine Learning (ICML). Vol. 70, pp: 1321-1330, 2017. https://arxiv.org/abs/1706.04599
VIII. C. Wei, W. Wang, W. Yang, J. Liu. : ‘Deep Retinex Decomposition for Low-Light Enhancement’. Proceedings of the British Machine Vision Conference (BMVC). 2018. https://arxiv.org/abs/1808.04560
IX. Canny, J. : ‘A Computational Approach to Edge Detection’. IEEE Transactions on Pattern Analysis and Machine Intelligence. Vol. PAMI-8(6), pp: 679-698, 1986. 10.1109/TPAMI.1986.4767851
X. D. Levi, L. Gispan, N. Giladi, E. Fetaya. : ‘Evaluating and Calibrating Uncertainty Prediction in Regression Tasks’. Sensors. Vol. 22(15), Article No. 5540, 2022. 10.3390/s22155540

XI. D. R. Martin, C. C. Fowlkes, J. Malik. : ‘Learning to Detect Natural Image Boundaries Using Local Brightness, Color, and Texture Cues’. IEEE Transactions on Pattern Analysis and Machine Intelligence. Vol. 26(5), pp: 530-549, 2004. 10.1109/TPAMI.2004.1273918
XII. H. Wang, et al. : ‘Low-Light Image Enhancement with Normalizing Flow’. Proceedings of the AAAI Conference on Artificial Intelligence (AAAI). 2022. https://arxiv.org/abs/2109.05923
XIII. J. Anaya, A. Barbu. : ‘RENOIR – A Dataset for Real Low-Light Image Noise Reduction’. Journal of Visual Communication and Image Representation. Vol. 51, pp: 144-154, 2018. arXiv:1409.8230, https://arxiv.org/abs/1409.8230
XIV. J. Liang, J. Cao, G. Sun, K. Zhang, L. Van Gool, R. Timofte. : ‘SwinIR: Image Restoration Using Swin Transformer’. Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops (ICCVW). pp: 1833-1844, 2021. 10.1109/ICCVW54120.2021.00210
XV. K. Dabov, A. Foi, V. Katkovnik, K. Egiazarian. : ‘Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering’. IEEE Transactions on Image Processing. Vol. 16(8), pp: 2080-2095, 2007. 10.1109/TIP.2007.901238
XVI. K. Zhang, W. Zuo, L. Zhang. : ‘FFDNet: Toward a Fast and Flexible Solution for CNN-Based Image Denoising’. IEEE Transactions on Image Processing. Vol. 27(9), pp: 4608-4622, 2018. 10.1109/TIP.2018.2839891
XVII. K. Zhang, W. Zuo, S. Gu, L. Zhang. : ‘Learning Deep CNN Denoiser Prior for Image Restoration’. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp: 3929-3938, 2017. https://arxiv.org/abs/1704.03264
XVIII. K. Zhang, W. Zuo, Y. Chen, D. Meng, L. Zhang. : ‘Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising’. IEEE Transactions on Image Processing. Vol. 26(7), pp: 3142-3155, 2017. 10.1109/TIP.2017.2662206
XIX. L. Chen, X. Chu, X. Zhang, J. Sun. : ‘Simple Baselines for Image Restoration’. Proceedings of the European Conference on Computer Vision (ECCV). pp: 17-33, 2022. 10.1007/978-3-031-20071-7_2
XX. M. Elad, M. Aharon. : ‘Image Denoising via Sparse and Redundant Representations over Learned Dictionaries’. IEEE Transactions on Image Processing. Vol. 15(12), pp: 3736-3745, 2006. 10.1109/TIP.2006.881969
XXI. P. Charbonnier, L. Blanc-Féraud, G. Aubert, M. Barlaud. : ‘Two Deterministic Half-Quadratic Regularization Algorithms for Computed Imaging’. Proceedings of the IEEE International Conference on Image Processing (ICIP). Vol. 2, pp: 168-172, 1994. 10.1109/ICIP.1994.413553
XXII. R. Zhang, P. Isola, A. A. Efros, E. Shechtman, O. Wang. : ‘The Unreasonable Effectiveness of Deep Features as a Perceptual Metric’. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp: 586-595, 2018. https://arxiv.org/abs/1801.03924
XXIII. S. W. Zamir, et al. : ‘Learning Enriched Features for Real Image Restoration and Enhancement’. Proceedings of the European Conference on Computer Vision (ECCV). pp: 492-511, 2020. https://arxiv.org/abs/2003.06792
XXIV. S. W. Zamir, et al. : ‘Multi-Stage Progressive Image Restoration’. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp: 14821-14831, 2021. https://arxiv.org/abs/2102.02808
XXV. S. W. Zamir, et al. : ‘Restormer: Efficient Transformer for High-Resolution Image Restoration’. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp: 5728-5739, 2022. https://arxiv.org/abs/2111.09881
XXVI. T. Brooks, B. Mildenhall, T. Xue, J. Chen, D. Sharlet, J. T. Barron. : ‘Unprocessing Images for Learned Raw Denoising’. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp: 11036-11045, 2019. https://arxiv.org/abs/1811.11127
XXVII. X. Chen, L. Chen, X. Chu, J. Sun. : ‘HINet: Half Instance Normalization Network for Image Restoration’. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 2021. https://arxiv.org/abs/2105.06086
XXVIII. X. Fu, D. Zeng, Y. Huang, Y. Liao, X. Ding, J. Paisley. : ‘A Fusion-Based Enhancing Method for Weakly Illuminated Images’. Signal Processing. Vol. 129, pp: 82-96, 2016. 10.1016/j.sigpro.2016.05.031
XXIX. X. Mao, C. Shen, Y.-B. Yang. : ‘Image Restoration Using Very Deep Convolutional Encoder-Decoder Networks with Symmetric Skip Connections’. Advances in Neural Information Processing Systems (NeurIPS). pp: 2802-2810, 2016. https://arxiv.org/abs/1603.09056
XXX. Y. Cai, K. Xu, Y. Li, J. Li. : ‘Learning a Deep Single Image Contrast Enhancer from Multi-Exposure Images’. IEEE Transactions on Image Processing. Vol. 27(4), pp: 2049-2062, 2018. 10.1109/TIP.2018.2794218
XXXI. Y. Chen, T. Pock. : ‘Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restoration’. IEEE Transactions on Pattern Analysis and Machine Intelligence. Vol. 39(6), pp: 1256-1272, 2017. 10.1109/TPAMI.2016.2596743
XXXII. Y. Jiang, X. Gong, D. Liu, Y. Cheng, C. Fang, X. Shen. : ‘EnlightenGAN: Deep Light Enhancement without Paired Supervision’. IEEE Transactions on Image Processing. Vol. 30, pp: 2340-2349, 2021. 10.1109/TIP.2021.3058764
XXXIII. Y. Zhang, X. Guo, J. Ma, W. Liu, J. Zhang. : ‘Beyond Brightening Low-Light Images’. International Journal of Computer Vision. Vol. 129, pp: 1013-1037, 2021. 10.1007/s11263-020-01407-x
XXXIV. Z. Liu, et al. : ‘Swin Transformer: Hierarchical Vision Transformer Using Shifted Windows’. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). pp: 10012-10022, 2021. 10.1109/ICCV48922.2021.00986
XXXV. Z. Tu, et al. : ‘MAXIM: Multi-Axis MLP for Image Processing’. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp: 5769-5780, 2022. https://arxiv.org/abs/2201.02973
XXXVI. Z. Wang, A. C. Bovik, H. R. Sheikh, E. P. Simoncelli. : ‘Image Quality Assessment: From Error Visibility to Structural Similarity’. IEEE Transactions on Image Processing. Vol. 13(4), pp: 600-612, 2004. 10.1109/TIP.2003.819861
XXXVII. Z. Wang, X. Cun, J. Bao, J. Liu. : ‘Uformer: A General U-Shaped Transformer for Image Restoration’. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp: 17683-17693, 2022. https://arxiv.org/abs/2106.03106

View Download