3.2 Transformaciones de Lorentz
3.2.6 Ejercicio
A. Fusion Scheme
The notations used in this section are as follows: A, B, R represent two source images and final fused image, respectively. C= (A, B, R). LFSC indicates the low frequency subband (LFS) of image C. HFSC
g,h indicates
the high frequency subband (HFS) of image C at scale g and direction h. (i, j) denotes spatial location, thus LFSC (i, j), HFSC
g,h (i, j) denote coefficients located at (i, j) of
low frequency and high frequency subband, respectively.
Step B.
We use maximum selection rule (MSR) [6] to select low frequency coefficients of LFSR from LFSA and LFSB
( )
A( )( )
A( )( )
B( )( )
R A B B LFS , , LFS , LFS , LFS , LFS , , LFS , < LFS , i j i j i j i j i j i j i j ≥ = as follow: (1) The coefficients of HFS of source images are selected by using SCM. As human vision system are sensitive to features such as edges, contours etc., so instead of using SCM directly, spatial frequency (SF) is considered as the gradient features of images to motivate SCM networks [25]. The SF is defined as , , , 2 , , 2 , , 1, , , 1 , ( ) ( ) g h g h g h g h g h i j i j i j i j i j i M j N S I I− I I − ∈ ∈ =∑
− + − (2) where Sg,h,i,j denotes the SF of the pixel that located at (i,j)
on scale g and direction h, and Ig,h,
i,j denotes the
coefficients of the pixel that located at (i,j) on scale g and direction h respectively.
SF in each high frequency subbands are inputted to SCM to motivate neurons and generate pulse of neurons as follow:
( )
(
)
, , , , , , , , , , , , , , , n n 1 ( 1) g h g h g h g h g h g h i j i j i j i j k l k l i j k l U = fU − +S∑
W Y n− +S (3)( )
(
)
, , , , n , n 1 , ( 1) g h g h g h i j i j i j E =gE − +hY n− (4)( )
( )
(
)
(
, ,)
, , , , 1, 1/ 1 exp n n 0.5 ( ) 0, o therwise g h g h i j i j g h i j U E Y n γ + − − > = (5)( )
(
)
, , , ,g h n ,g h n 1 ,g h( ) i j i j i j T =T − +Y n (6) where SF Sg,hi,j is set as feeding input of SCM, U g,h i,j(n) is
internal activity, n denotes iteration times. Yg,h i,j (n) is
output, Eg,h
i,j (n) is dynamic threshold, W g,h
ij,kl is synaptic
weight matrix applied to the linking field, f and g are decay constants, and h is threshold magnitude coefficient. As a typical neuronal nonlinear transform function, the Sigmoid function [26] is applied in SCM to improve performance, which helps make output reachable. γ is a parameter of Sigmoid function. The nonlinearity of Sigmoid function can be used to generate pulse. Sigmoid curve has an “S” shape, with its slope increasing as γ increases. If
Algorithm Yg,h
i,j(n) is equal to 1, it means the neuron
will generate a pulse, or we can say one firing occurs. The sum of Yg,h
i,j in n iteration (namely the firing times) is
defined as Tg,h
i,j (n) to represent the image information.
Rather than Yg,h
i,j (n), researchers often analyze T g,h i,j (n),
because neighboring coefficients with similar features representing similar firing times in a given iteration times. In this paper, we set firing times Tg,h
i,j (n) as criteria to
select coefficients of high frequency subbands.
1). Decompose source images A and B by NSCT to get low frequency and high frequency subbands coefficients of each image.
2). Select coefficients of LFSR
3). Calculate SF as described in formula (2) by using overlapping window on coefficients of HFS.
by using formula (1).
4). Input SF of each HFS into SCM and generate pulse of neurons with formula (3) ~ (5). And then compute the firing times Tg,h
i,j(n) by formula (6).
5). Fuse coefficients of each HFS by the following rules:
( )
,( )( )
,, ,( )
,, ,( )
, , HFS , , n n HFS , HFS , , ot herwise A g h A g h B g h i j i j R g h B g h i j T T i j i j ≥ = (7)6). Apply inverse NSCT on the fused LFS and HFS to get the final fused image.
It is important to perform statistical assessment of the quality of different fusion techniques along with the visual assessment due to the widespread use of multi-sensor and multi-spectral images in medical diagnosis. Therefore, image quality evaluation tools are required to compare the results achieved by different fusion techniques. In this paper, quantitative assessment of different image fusion algorithms are compared using following evaluation criteria, which have been proved to be effective to a great degree [2].
The notations used in this section are defined as : A and B are source images, R the final fused image, m × n the size of the image that has L grey levels; f (i, j) denotes grey value of pixel (i, j). Then the above 4 indices are mathematically described as:
1 1 ,R , 0 0 1 1 , , 0 0 ( , )log(( ( , )) / ( ( ) ( ))) M I = _ _ ( , )log(( ( , )) / ( ( ) ( ))) _ _ L L A A R A R i k L L B R B R B R j k P i k P i k P i P k IE A IE B P j k P j k P j P k IE A IE B − − = = − − = = + + +
∑ ∑
∑ ∑
1. Mutual information (MI):
(8) where P(i) indicates probability of pixels whose grey value amount to i; PA,R(i, k) and PB,R
2. Standard deviation (SD):
(j, k) are the normalized grey histogram between A and R and the normalized grey histogram between B and R, respectively. IE_A and IE_B denote the information entropy (IE) of image A and B. MI [27] can be used to measure amount of information transferred from source images to final fused image. Fusion performance would be better and better with MI increasing.
2 1 1 1 1 1 m n ( ( , ) 1 m n ( , )) i j i j S D f i j f i j m n = = m n = = = − ×
∑∑
×∑∑
(9) SD indicates deviation degree between grey values of pixels and the average one of the fused image.3. Energy of laplacian (EOL):
-1 -1 i 2 j 2 2 (- ( -1, -1) - 4 ( -1, ) - ( -1, +1) - 4 ( , -1) + 2 0 ( , ) - 4 ( , +1) - ( +1, -1) - 4 ( +1, ) - ( +1, +1) ) m n EOL f i j f i j f i j f i j f i j f i j f i j f i j f i j = = =
∑∑
(10) EOL is one of the useful indices to describe clarity ofimage. 4. QAB/F: AF BF AB/F 1 1 1 1 (Q ( , ) ( , ) Q ( , ) ( , )) Q = ( ( , ) ( , )) N M A B n m N M A B n m n m n m n m n m n m n m ω ω ω ω = = = = + +
∑ ∑
∑ ∑
(11) QAB/F [28] is proposed by C.S.Xydeas et al. as an objective image fusion performance measure. QAF(n,m)= QAF g (n,m) Q AF α (n,m). QAF g (n,m) and Q AF
α (n,m) are the edge
strength and orientation preservation values, respectively. QBF
(n,m) is similarly computed. ωA(n,m) and ωB(n,m) reflect the importance of QAF
g (n,m) and Q AF α (n,m),
respectively. The dynamic range of QAB/F
(n,m) is [0,1]. To evaluate the performance of the proposed image fusion approach, four different groups of human brain images are considered (see Figure. 3(a, b), Figure. 4(a, b), Figure. 5(a, b), Figure. 6(a, b)).The four groups of source images come from the website of the Atlas project of Harvard Medical School [29]. Figure.3 (a) and Figure.4 (a) are original CT image, Figure.3 (b) and Figure.4 (b) are original MRI image, Figure.5 (a) is original coronal FDG image, Figure.5 (b) is original coronal MR-T1 image, Figure.6 (a) is original MR-T1 image, Figure.6 (b) is original MR-T2 image. CT image shows structures of bone, while MR image shows areas of soft tissue. All images have 256-level gray scale. It can be seen that due to various imaging principle and environment, the source images with different modality contain complementary information.
For all these image groups, results of proposed fusion framework are compared with Averaging method, PCA method, discrete wavelet transform (DWT) with DBSS (2, 2), Laplacian pyramid (LP), morphological pyramid (MP). Parameters of these methods are set by: pyramid level = 4, selection rules: high-pass = select max, lowpass = average [11]. The visual comparison for fused images according to different fusion algorithms are shown in
Figure.3 (c–h), Figure.4 (c–h), Figure.5 (c–h), Figure.6 (c–h).
IV.EVALUATION AND ANALYSIS
From the fusion results displayed in Figure 3-6, it is clear that the Averaging and PCA algorithm give lose too many image details and provide poor fusion results compared with other four algorithms. This is because both of them have no scale selectivity. This limitation is modified in DWT, LP and MP. Morphological pyramid (MP) provides satisfactory fusion result, but it always brings fake image information and result in block effect. The remaining DWT, Laplacian pyramid (LP) and our proposed method achieved similar fusion effect. From subjective observation, we can see that the proposed algorithm is effective in multimodal medical image fusion. One of the reasons behind the performance of the proposed method is the human visual characteristics of SCM, which brings high contrast and more informative details to fused images.
Statistically, the objective performance evaluation and comparison among existing and proposed algorithms are depicted in Table 1. From experimental data we can see that the values of QABF of our proposed methods are the best in image group 1, group 2 and group 4, the values of MI of our proposed methods are the best in image group 2 and group 3. Though the other values of our method are not the best, it still got the second high values and outperformed the other 4 methods in each group. From table, we can easily conclude that the proposed algorithm can preserve high spatial resolution characteristics with less spectral distortion. The objective evaluation and comparison we discussed above verified that the proposed method is an effective fusion method for multimodal medical image fusion.
Figure.3. Fusion results of image group 1: (a) original CT image; (b) original MRI image; Fused image using (c) Averaging method, (d) PCA, (e) DWT, (f) Laplacian pyramid (LP), (g) morphological pyramid (MP), (h) our proposed method. All original images are from the website of the Atlas
project of Harvard Medical School [29].
Figure.4. Fusion results of image group 3: (a) original CT image; (b) original MRI image; fused image using (c) Averaging, (d) PCA, (e) DWT, (f) LP, (g) MP, (h) our proposed method.
Figure.5. Fusion results of image group 2: (a) original coronal FDG image; (b) original coronal MR-T1 image; Fused image using (c) Averaging, (d) PCA, (e) DWT, (f) LP, (g) MP, (h) our proposed method.
Figure.6. Fusion results of image group 4: (a) original MR-T1 image; (b) original MR-T2 image; Fused image using (c) Averaging method, (d) PCA, (e) DWT, (f) Laplacian pyramid (LP), (g) morphological pyramid (MP), (h) our proposed method.
V
Table.1. Comparison of six fusion method of three image groups
source Images
evaluation
indices Averaging PCA DWT LP MP Proposed
image group 1 MI 3.5955 4.2999 1.3981 1.7041 2.2017 3.8106 SD 7.8904 8.4250 7.9004 7.8479 7.9643 8.0021 EOL 0.2306 0.3273 0.9520 1.0039 3.0433 1.0028 QAB/F 0.4264 0.6549 0.6339 0.7442 0.7087 0.7463 image group 2 MI 3.1889 3.1936 1.4811 1.6714 1.9625 3.2123 SD 9.3425 9.6711 9.4424 9.4346 9.6130 9.6308 EOL 0.3805 0.5236 1.2423 1.2797 1.4310 1.2955 QAB/F 0.4398 0.6486 0.6774 0.7020 0.7051 0.7097 image group 3 MI 2.4207 2.6009 1.9205 2.1461 2.3388 2.6808 SD 11.2629 10.5719 10.4373 9.9959 9.4862 10.6887 EOL 1.2571 1.5205 4.3872 4.4333 5.5133 4.0921 QAB/F 0.3862 0.4805 0.5405 0.5832 0.5530 0.5766 image group 4 MI 2.6184 2.8710 2.2257 2.3604 2.3740 2.6334 SD 10.6775 10.6557 10.4352 10.5081 10.2899 10.5290 EOL 0.2330 0.2996 0.8110 0.9174 1.2899 0.9977 QAB/F 0.3591 0.4359 0.4426 0.4965 0.4277 0.5190 .CONCLUSIONS
In this paper, we proposed a new medical image fusion algorithm based on NSCT and SCM. The flexible multi-resolution, anisotropy, and directional expansion characteristics of NSCT are associated with global coupling and pulse synchronization features of SCM. Considering the human visual system characteristics, two different fusion rules are used to fuse the low and high frequency sub-bands respectively. Spatial frequency is applied to motivate SCM network rather than using coefficients value directly. The efficiency of the proposed algorithm is achieved by the comparison with existing fusion methods. The statistical comparisons prove the effectiveness of the proposed fusion method.
ACKNOWLEDGEMENTS
The authors would like to thank the anonymous reviewers and editors for their invaluable suggestions. This work was jointly supported by the National Natural Science Foundation of China (No. 61175012, 61162021, 61201422), Science and Technology Innovation Project of Ministry of Culture No [2011] 820, and Innovative Team Subsidize of Northwest University for Nationalities.
REFERENCES
[1] Wong, Alexander, and William Bishop. "Efficient least squares fusion of MRI and CT images using a phase congruency model." Pattern Recognition Letters, Vol. 29, No.3, pp. 173-180, 2008.
[2] Bhatnagar, Gaurav, Q. M. Jonathan Wu, and Zheng Liu. "Human visual system inspired multi-modal medical image fusion framework." Expert Systems with Applications, 2012.
[3] Anderson, C. H. U.S. Patent No. 4,718,104. Washington, DC: U.S. Patent and Trademark Office, 1998.
[4] Burt, Peter J. "A gradient pyramid basis for pattern-selective image fusion." Proceedings of the Society for Information Display, pp. 467-470, 1992.
[5] Burt, Peter, and Edward Adelson. "The Laplacian pyramid as a compact image code." IEEE Transactions on Communications, Vol.31, No.4, pp. 532-540, 1983. [6] Li, Hui, B. S. Manjunath, and Sanjit K. Mitra.
"Multisensor image fusion using the wavelet transform." Graphical models and image processing, Vol.57, No.3, pp. 235-245, 1995.
[7] Rockinger, Oliver. "Image sequence fusion using a shift-invariant wavelet transform." Image Processing, 1997. International Conference on Proceedings, Vol. 3, IEEE, 1997.
[8] Toet, A. "A morphological pyramidal image decomposition." Pattern Recognition Letters, Vol.9, No.4, pp. 255-261, 1989.
[9] Toet, Alexander. "Image fusion by a ratio of low-pass pyramid." Pattern Recognition Letters, Vol.9, No.4, pp. 245-253, 1989.
[10] Toet, Alexander, Lodewik J. Van Ruyven, and J. Mathee Valeton. "Merging thermal and visual images by a contrast pyramid." Optical Engineering, Vol.28, No.7, pp. 287789-287789, 1989.
[11] Wang Zhaobin, and Yide Ma. "Medical image fusion using< i> m</i>-PCNN." Information Fusion, Vol.9, No.2, pp. 176-185, 2008.
[12] Zhang, Zhong, and Rick S. Blum. "A categorization of multiscale-decomposition-based image fusion schemes with a performance study for a digital camera application." Proceedings of the IEEE, Vol.87, No.8, pp. 1315-1326, 1999.
[13] Da Cunha, Arthur L., Jianping Zhou, and Minh N. Do. "The nonsubsampled contourlet transform: theory, design, and applications." IEEE Transactions on Image Processing, Vol.15, No.10, pp. 3089-3101, 2006.
[14] Johnson, John L., and Mary Lou Padgett. "PCNN models and applications." IEEE Transactions on Neural Networks, Vol.10, No.3, pp. 480-498, 1999.
[15] Zhan, Kun, Hongjuan Zhang, and Yide Ma. "New spiking cortical model for invariant texture retrieval and image processing." IEEE Transactions on Neural Networks, Vol.20, No.12, pp. 1980-1986, 2009.
[16] Kavitha, C. T., C. Chellamuthu, and R. Rajesh. "Multimodal Medical Image Fusion Using Discrete Ripplet Transform and Intersecting Cortical Model." Procedia Engineering, Vol.38, pp. 1409-1414, 2012. [17] Chai, Y., H. F. Li, and J. F. Qu. "Image fusion scheme
using a novel dual-channel PCNN in lifting stationary wavelet domain." Optics Communications, Vol.283, No.19, pp. 3591-3602, 2010.
[18] Qiguang, Miao, Shi Cheng, and Xu Pengfei. "A novel algorithm of image fusion based on shearlets and PCNN." Neurocomputing, 2012.
[19] Yang, Shuyuan, Min Wang, and Licheng Jiao. "Contourlet hidden Markov Tree and clarity-saliency driven PCNN based remote sensing images fusion." Applied Soft Computing, Vol.12, No.1, pp. 228-237, 2012.
[20] Liu, Shengpeng, and Yong Fang. "Infrared image fusion algorithm based on contourlet transform and improved pulse coupled neural network." JOURNAL OF INFRARED AND MILLIMETER WAVES-CHINESE EDITION-, Vol. 26, No.3, pp. 217, 2007.
[21] Qu, Xiao-Bo, et al. "Image fusion algorithm based on spatial frequency-motivated pulse coupled neural networks in nonsubsampled contourlet transform domain." Acta Automatica Sinica. Vol.34, No.12, pp. 1508-1514, 2008.
[22] Xin, Guojiang, et al. "Multi-focus image fusion based on the nonsubsampled contourlet transform and dual-layer PCNN model." Information Technology Journal, Vol.10, No.6, pp. 1138-1149, 2011.
[23] Das, Sudeb, and Malay Kumar Kundu. "NSCT-based multimodal medical image fusion using pulse-coupled neural network and modified spatial frequency." Medical & biological engineering & computing. Vol.50, No.10, pp. 1105-1114, 2012.
[24] Do, Minh N., and Martin Vetterli. "The contourlet transform: an efficient directional multiresolution image representation." IEEE Transactions on Image Processing, Vol.14, No.12, pp. 2091-2106, 2005.
[25] Eskicioglu, Ahmet M., and Paul S. Fisher. "Image quality measures and their performance." IEEE Transactions on Communications, Vol.43, No.12, pp. 2959-2965, 1995. [26] Forgac, R., and I. Mokris. "Feature generation improving
by optimized PCNN." Applied Machine Intelligence and Informatics, 2008. SAMI 2008. 6th International Symposium on. IEEE, 2008.
[27] Qu, Guihong, Dali Zhang, and Pingfan Yan. "Information measure for performance of image fusion." Electronics letters. Vol.38, No.7, pp. 313-315, 2002.
[28] Xydeas, C. S., and V. Petrovic. "Objective image fusion performance measure." Electronics Letters. Vol.36, No.4, pp. 308-309, 2000.
[29] http://www.med.harvard.edu/AANLIB
Nianyi Wang received the B.S. degree in computer science
from Lanzhou University, Gansu, China in 2002, and received the M.S. degree in computer application technology from Lanzhou University in 2007. He is currently a lecturer in the School of Mathematics and Computer Science Institute, Northwest University for Nationalities. He is currently pursuing the Ph.D. degree in radio physics at Lanzhou University. His current research interests include artificial neural networks, image processing, and pattern recognition.
Yide Ma received the B.S. and M.S. degrees in radio
technology from Chengdu University of Engineering Science and Technology, Sichuan, China, in 1984 and 1988, respectively. He received the Ph.D. degree from the Department of Life Science, Lanzhou University, Gansu, China, in 2001. He is currently a Professor in the School of Information Science and Engineering, Lanzhou University. He has published more than 50 papers in major journals and international conferences and several textbooks, including Principle and Application of Pulse Coupled Neural Network (Beijing: Science Press, 2006), and Principle and Application of Microcomputer (Beijing: Science Press, 2006). His current research interests include artificial neural networks, digital image processing, pattern recognition, digital signal processing, and computer vision.
Kun Zhan received the B.S. degree in electronic information
science and technology from Lanzhou University, China, in 2005. He received the Ph.D. at the same university in 2010. Currently, He works in Lanzhou University. His main research interests are image processing and neural computation.
Min Yuan received the B.S. degree in electronic information
science and technology from Lanzhou University, China in 2002, and received the M.S. degree in Communication and information system from Lanzhou University in 2005. She is currently a lecturer in the School of Information Science and Engineering, Lanzhou University. She is currently pursuing the Ph.D. degree in radio physics at Lanzhou University. Her research interests include compressed sensing, magnetic resonance imaging, and sparse signal and image processing, wavelets and other multi-scale transforms.