Project

Kazuki Naganuma and Shunsuke Ono

This paper proposes a foreground-background separation (FBS) method with a novel foreground model based on convolutional sparse representation (CSR). In order to analyze the dynamic and static components of videos acquired under undesirable conditions, such as hardware, environmental, and power limitations, it is essential to establish an FBS method that can handle videos with low frame rates and various types of noise. Existing FBS methods have two limitations that prevent us from accurately separating foreground and background components from such degraded videos. First, they only capture either data-specific or general features of the components. Second, they do not include explicit models for various types of noise to remove them in the FBS process. To this end, we propose a robust FBS method with a CSR-based foreground model. This model can adaptively capture specific spatial structures scattered in imaging data. Then, we formulate FBS as a constrained multiconvex optimization problem that incorporates CSR, functions that capture general features, and explicit noise characterization functions for multiple types of noise. Thanks to these functions, our method captures both data-specific and general features to accurately separate the components from various types of noise even under low frame rates. To obtain a solution of the optimization problem, we develop an algorithm that alternately solves its two convex subproblems by newly established algorithms. Experiments demonstrate the superiority of our method over existing methods using two types of degraded videos: infrared and microscope videos.

• 陽に数理モデル化しやすい画像全体に共通する特徴（共通特徴：general features）と各画像に固有に表れる特徴（固有特徴：specific featrues）を同時に捉えることができる前景背景分離法（FBS）の手法を提案

• 畳み込みスパース表現により、適応的に固有特徴を捉える変換を実現

• 共通特徴、固有特徴、そして陽にモデル化した複数のノイズの特徴を捉える関数・制約を含めた最適化問題を設計

• 最適化問題を解くためのアルゴリズムを確立

[1] E. J. Candes, X. Li, and J. Wright, “Robust principal component analysis?” J. ACM, vol. 58, no. 3, pp. 11:1–11:37, Jun. 2011.

[2] Z. Yang, L. Fan, Y. Yang, Z. Yang, and G. Gui, “Generalized nuclear norm and Laplacian scale mixture based low-rank and sparse decomposition for video foreground-background separation,” Signal Process., vol. 172, p. 107527, 2020.

[3] X. Cao, L. Yang, and X. Guo, “Total variation regularized RPCA for irregularly moving object detection under dynamic background,” IEEE Trans. Cybern., vol. 46, no. 4, pp. 1014–1027, Apr. 2016.

[4] B. E. Moore, C. Gao, and R. R. Nadakuditi, “Panoramic robust PCA for foreground-background separation on noisy, free-motion camera video,” IEEE Trans. Comput. Imag., vol. 5, no. 2, pp. 195–211, Jun. 2019.

[5] B. Shen, W. Xie, and Z. J. Kong, “Smooth robust tensor completion for background/foreground separation with missing pixels: Novel algorithm with convergence guarantee,” J. Mach. Learn. Res., vol. 23, no. 217, pp. 1–40, Jul. 2022.

[6] B. Shen, R. R. Kamath, H. Choo, and Z. Kong, “Robust tensor decomposition based background/foreground separation in noisy videos and its applications in additive manufacturing,” IEEE Trans. Autom. Sci. Eng., vol. 20, no. 1, pp. 583–596, 2023.

[7] Y.-C. Miao, X.-L. Zhao, J.-L. Wang, X. Fu, and Y. Wang, “Snapshot compressive imaging using domain-factorized deep video prior,” IEEE Trans. Comput. Imag., vol. 10, pp. 93–102, 2024.

@misc{naganuma2025robust,

      title={Robust Foreground-Background Separation for Severely-Degraded Videos Using Convolutional Sparse Representation Modeling}, 

      author={Kazuki Naganuma and Shunsuke Ono},

      year={2025},

      eprint={2506.17838},

      archivePrefix={arXiv},

      primaryClass={eess.IV}

}

• arXiv

• MATLAB code

• GitHub