Multi-focus image fusion based on block matching in 3D transform domain

doi:10.21629/JSEE.2018.02.21

Journal of Systems Engineering and Electronics ›› 2018, Vol. 29 ›› Issue (2): 415-428.doi: 10.21629/JSEE.2018.02.21

• Software Algorithm and Simulation • Previous Articles Next Articles

Multi-focus image fusion based on block matching in 3D transform domain

Dongsheng YANG^1,²(), Shaohai HU^1,^2,*(), Shuaiqi LIU³(), Xiaole MA^1,²(), Yuchao SUN⁴()

¹ Institute of Information Science, Beijing Jiaotong University, Beijing 100044, China
² Beijing Key Laboratory of Advanced Information Science and Network Technology, Beijing 100044, China
³ College of Electronic and Information Engineering, Hebei University, Baoding 071002, China
⁴ The Third Research Institute, China Electronics Technology Group Corporation, Beijing 100015, China

Received:2017-04-05 Online:2018-04-26 Published:2018-04-27
Contact: Shaohai HU E-mail:dsyang@bjtu.edu.cn;shhu@bjtu.edu.cn;shdkj-1918@163.com;maxiaole@bjtu.edu.cn;13120337@bjtu.edu.cn
About author:YANG Dongsheng was born in 1991. He received his B.S. degree in computer science at the School of Computer and Information Technology, Beijing Jiaotong University in 2014. At present, he is pursuing his M.S. degree in information and signal processing at the Institute of Information Science, Beijing Jiaotong University. Currently, his research interests include image fusion, and image denoising. E-mail: dsyang@bjtu.edu.cn|HU Shaohai was born in 1964. He received his B.S. and M.S. degrees in the Department of Electronic Engineering from Beihang University in 1985 and 1988, respectively. He received his Ph.D. degree in Institute of Information Science from Beijing Jiaotong University in 1991 and has been a professor of Institute of Information Science since 2008. He has co-authored more than 100 journal articles and conference proceedings, and has published three books in his research area. His research interests lie in the broad area of signal processing and information fusion, including image fusion, image denosing and sparse representation. E-mail: shhu@bjtu.edu.cn|LIU Shuaiqi was born in 1986. He received his B.S. degree in the Department of Information and Computer Science, Shandong University of Science and Technology in 2009. He received his Ph.D. degree in Institute of Information Science from Beijing Jiaotong University in 2014 and has been a teacher of Hebei University since 2014. His research interests include image processing and human-computer. E-mail: shdkj-1918@163.com|MA Xiaole was born in 1991. She received his B.S. degree in communication engineering at the College of Electronic and Information Engineering, Hebei University. At present, she is pursuing her Ph.D. degree in information and signal processing at the Institute of Information Science, Beijing Jiaotong University. Currently, her research interests include image fusion, and image denoising. E-mail: maxiaole@bjtu.edu.cn|SUN Yuchao was born in 1989. He received his B.S. degree and M.S. degree from the North University of China in 2013 and from the Institute of Information Science, Beijing Jiaotong University in 2016, respectively. Currently, he is a researcher at the Third Research Institute of China Electronics Technology Group Corporation. His work focuses on signal processing. E-mail: 13120337@bjtu.edu.cn
Supported by:
the National Natural Science Foundation of China(61572063);the National Natural Science Foundation of China(61401308);the Fundamental Research Funds for the Central Universities(2016YJS039);the Natural Science Foundation of Hebei Province(F2016201142);the Natural Science Foundation of Hebei Province(F2016201187);the Natural Social Foundation of Hebei Province(HB15TQ015);the Science Research Project of Hebei Province(QN2016085);the Science Research Project of Hebei Province(ZC2016040);the Natural Science Foundation of Hebei University(2014-303);This work was supported by the National Natural Science Foundation of China (61572063; 61401308), the Fundamental Research Funds for the Central Universities (2016YJS039), the Natural Science Foundation of Hebei Province (F2016201142; F2016201187), the Natural Social Foundation of Hebei Province (HB15TQ015), the Science Research Project of Hebei Province (QN2016085; ZC2016040), and the Natural Science Foundation of Hebei University (2014-303)

Abstract

Abstract:

Fusion methods based on multi-scale transforms have become the mainstream of the pixel-level image fusion. However, most of these methods cannot fully exploit spatial domain information of source images, which lead to the degradation of image. This paper presents a fusion framework based on block-matching and 3D (BM3D) multi-scale transform. The algorithm first divides the image into different blocks and groups these 2D image blocks into 3D arrays by their similarity. Then it uses a 3D transform which consists of a 2D multi-scale and a 1D transform to transfer the arrays into transform coefficients, and then the obtained low- and high- coefficients are fused by different fusion rules. The final fused image is obtained from a series of fused 3D image block groups after the inverse transform by using an aggregation process. In the experimental part, we comparatively analyze some existing algorithms and the using of different transforms, e.g. non-subsampled Contourlet transform (NSCT), non-subsampled Shearlet transform (NSST), in the 3D transform step. Experimental results show that the proposed fusion framework can not only improve subjective visual effect, but also obtain better objective evaluation criteria than state-of-the-art methods.

Key words: image fusion, block matching, 3D transform, blockmatching and 3D (BM3D), non-subsampled Shearlet transform (NSST)

Dongsheng YANG, Shaohai HU, Shuaiqi LIU, Xiaole MA, Yuchao SUN. Multi-focus image fusion based on block matching in 3D transform domain[J]. Journal of Systems Engineering and Electronics, 2018, 29(2): 415-428.

Figures/Tables 16

Fig 1

Fig 2

Fig 3

Fig 4

Fig 5

Fig 6

Fig 7

Fig 8

Fig 9

Fig 10

Fig 11

Table 1

Fig 12

Fig 13

Table 2

Table 3

References 28

1	HAGHIGHAT M B A, AGHAGOLZADEH A, SEYEDARABI H. Multi-focus image fusion for visual sensor networks in DCT domain. Computers & Electrical Engineering, 2011, 37 (5): 789- 797.
2	ZHANG Z, BLUM R S. A categorization of multiscaledecomposition-based image fusion schemes with a performance study for a digital camera application. Proceedings of the IEEE, 1999, 87 (2): 1315- 1326.
3	PAJARES G, CRUZ J M D L. A wavelet-based image fusion tutorial. Pattern Recognition, 2004, 37 (9): 1855- 1872.
4	CANDES E J. Ridgelets: theory and applications. Stanford, USA: Stanford University, 1998.
5	COHEN R A, SCHUMAKAR L L. Curves and surfaces. Nashville: Vanderbilt University Press, 2000.
6	DO M N, VETTERLI M. The Contourlet transform: an efficient directional multi-resolution image representation. IEEE Trans. on Image Processing, 2005, 14(12): 2091-2106.
7	ZHANG Q, GUO B L. Multifocus image fusion using the nonsubsampled contourlet transform. Signal Processing, 2009, 89 (7): 1334- 1346.
8	WANG J, PENG J Y, FENG X Y, et al. Image fusion with nonsubsampled contourlet transform and sparse representation. Journal of Electronic Imaging, 2013, 22 (4): 043019.
9	GUO K, LABATE D. Optimally sparse multidimensional representation using shearlets. SIAM Journal on Mathematical Analysis, 2007, 39 (1): 298- 318.
10	NUNEZ J, OTAZU X, FORS O, et al. Multiresolution-based image fusion with additive wavelet decomposition. IEEE Trans. on Geoscience and Remote Sensing, 2002, 37(3): 1204-1211.
11	GENG P, WANG Z Y, ZHANG Z G, et al. Image fusion by pulse couple neural network with shearlet. Optical Engineering, 2010, 51 (6): 067005- 1.
12	DABOV K, FOI A, KATKOVNIK V, et al. Image denoising with block-matching and 3D filtering. Proc. of SPIE-IS & T Electronic Imaging: Algorithms and Systems V, 2006, 6064: 606414-1-606414-12.
13	DABOV K, FOI A, KATKOVNIK V, et al. Image denosing by sparse 3D transform-domain collaborative filtering. IEEE Trans. on Image Processing, 2007, 16(8): 2080-2095.
14	TIAN J, CHEN J, ZHANG C. Multispectral image fusion based on fractal features. Proceedings of SPIE, 2004, 5308, 824- 832.
15	BHATNAGAR G, WU Q M J, LIU Z. Directive contrast based multimodal medical image fusion in NSCT domain. IEEE Trans. on Multimedia, 2013, 15(5): 1014-1024.
16	KUMAR M, DASS S. A total variation-based algorithm for pixel-level image fusion. IEEE Trans. on Image Processing, 2009, 18(9): 2137-2143.
17	BUADES A, COLL B, MOREL J M. A review of image denoising algorithms, with a new one. SIAM Journal on Multiscale Modeling and Simulation, 2005, 4 (2): 490- 530.
18	DO M N, VETTERLI M. Wavelet-based texture retrieval using generalized Gaussian density and Kullback-Leibler distance. IEEE Trans. on Image Processing, 2002, 11(2): 146-158.
19	MACQUEEN J B. Some methods for classification and analysis of multivariate observations. Proc. of the 5th Berkeley Symposium on Mathematical Statistics and Probability, 1967: 281-297.
20	HÖPPNER F, KLAWONN F, KRUSE R, et al. Fuzzy cluster analysis. Chichester: Wiley, 1999.
21	GERSHO A. On the structure of vector quantizers. IEEE Trans. on Information Theory, 1982, 28(2): 157-166.
22	JIANG P, ZHANG Q, LI J, et al. Fusion algorithm for infrared and visible image based on NSST and adaptive PCNN. Laser and Infrared, 2014, 44 (1): 108- 112.
23	GULERYUZ O. Weighted overcomplete denoising. Proc. of the 7th Asilomar Conference on Signals, Systems and Computers, 2003, 2: 1992-1996.
24	LIU S, ZHU Z, LI H, et al. Multi-focus image fusion using self-similarity and depth information in nonsubsampled shearlet transform domain. International Journal of Signal Processing, Image Processing and Pattern Recognition, 2016, 9 (1): 347- 360.
25	QU G H, ZHANG D L, YAN P F. Information measure for performance of image fusion. Electronic Letters, 2002, 38 (7): 313- 315.
26	XYDEAS C S, PETROVI V. Objective image fusion performance measure. Electronics Letters, 2000, 36 (4): 308- 309.
27	WANG Z, BOVIK A C, SHEIK H R, et al. Image quality assessment: from error visibility to structural similarity. IEEE Trans. on Image Processing, 2004, 13(4): 600-612.
28	MIAO Q G, SHI C, XU P F, et al. Multi-focus image fusion algorithm based on shearlets. Chinese Optics Letters, 2011, 9 (4): 25- 29.

Fusion	Objective criteria
method	EN	STD	AVG	MI	${Q^{AB/F}}$	SSIM
BMDCT	7.028 1	46.794 1	3.070 6	5.423 4	0.469 6	0.856 3
BMDWT	7.039 9	47.310 8	3.707 0	7.089 9	0.718 9	0.895 8
BMNSCT	7.068 7	48.097 7	3.774 1	7.196 4	0.720 3	0.894 0
BMNSST	7.070 1	48.242 5	3.767 8	7.207 6	0.729 2	0.892 8

Fusion	Objective criteria
method	EN	STD	AVG	MI	${Q^{AB/F}}$	SSIM
DWT-MAX	7.102 1	44.908 7	4.088 9	6.432 2	0.710 0	0.894 5
NSCT-MAX	7.095 6	44.926 2	4.061 2	6.823 6	0.770 7	0.906 7
NSST-MAX	7.099 1	44.917 6	4.087 3	6.758 3	0.762 4	0.905 3
BMDWT-CMIS	7.103 3	45.105 6	4.081 4	6.625 3	0.757 5	0.902 4
BMNSCT-CMIS	7.105 2	45.252 1	4.201 3	6.861 4	0.759 6	0.908 6
BMNSST-CMIS	7.106 8	45.380 3	4.132 4	6.903 5	0.778 3	0.903 9

Fusion	Objective criteria
method	EN	STD	AVG	MI	${Q^{AB/F}}$	SSIM
DWT-MAX	7.034 4	40.367 4	2.886 7	6.414 6	0.612 6	0.898 2
NSCT-MAX	7.044 8	40.921 7	2.881 7	6.738 7	0.676 6	0.908 5
NSST-MAX	7.048 1	40.808 8	2.747 4	6.496 1	0.601 1	0.860 1
BMDWT-CMIS	7.058 4	40.347 8	2.756 3	6.653 9	0.646 7	0.899 2
BMNSCT-CMIS	7.080 5	41.029 7	2.907 1	6.721 8	0.685 3	0.909 0
BMNSST-CMIS	7.087 2	41.064 8	2.889 1	6.934 4	0.689 4	0.903 3

[1]	Jian WANG, Chunxia QIN, Xiufei ZHANG, Ke YANG, Ping REN. A multi-source image fusion algorithm based on gradient regularized convolution sparse representation [J]. Journal of Systems Engineering and Electronics, 2020, 31(3): 447-459.
[2]	Jian WANG, Ke YANG, Ping REN, Chunxia QIN, Xiufei ZHANG. Multi-source image fusion algorithm based on fast weighted guided filter [J]. Journal of Systems Engineering and Electronics, 2019, 30(5): 831-840.
[3]	Kun Liu1, Lei Guo, and Jingsong Chen. Contourlet transform for image fusion using cycle spinning [J]. Journal of Systems Engineering and Electronics, 2011, 22(2): 353-357.
[4]	Tao Tangfei, Ja Choon Koo & Hyouk Ryeol Choi. Improved block matching approach to fast disparity estimation? [J]. Journal of Systems Engineering and Electronics, 2009, 20(6): 1278-1285.

Multi-focus image fusion based on block matching in 3D transform domain

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

Share this article

Figures/Tables 16

References 28

Related Articles 4

Recommended Articles

Metrics

Comments