Hybrid Fusion-Based Background Segmentation in Multispectral Polarimetric Imagery
Abstract
:1. Introduction
2. Related Works
2.1. Fusion
2.2. Segmentation
3. Analysis of the MSPI Dataset
3.1. Description of the Dataset
3.2. Analysis of Spectral Reflectance
3.3. Analysis of Multiband the Dissimilarity Matrix
3.3.1. Pearson Correlation
3.3.2. Euclidean Distance
3.4. Analysis of Multiband Textural Features
1. | Mean is a measure of the spreading of the distribution from the mean value. | (13) | |
2. | Standard deviation is used to sharpen edges as the intensity level changes by a large value at the edge of an image. | (14) | |
3. | Energy is a measure of the homogeneity of the histogram. | (15) | |
4. | Skewness is a measure of the degree of the histogram’s asymmetry around the mean. | (16) | |
5. | Kurtosis is a measure of the histogram’s sharpness, that is, whether the data are peaked or flat relative to a normal distribution. | (17) |
1. | Contrast is a measure of the local variations present in an image, that is, it reflects the depth and smoothness of the image’s textural structure. | (19) | |
2. | Correlation is a measure of the gray-level linear dependence between the pixels at specified positions relative to each other, that is, it reflects the similarity of an image’s texture in the horizontal or vertical direction. | (20) | |
, | |||
3. | Angular Second Moment or Energy is a measure of the global homogeneity of an image. | (21) | |
4. | Homogeneity is a measure of the local homogeneity of an image. | (22) | |
5. | Entropy is a measure of a histogram’s uniformity, that is, it reflects the complexity of the textural distribution. | (23) |
4. Proposed Two-Fold BS
4.1. Overall Framework and Algorithm
Algorithm 1. Multiband Fusion and BS | ||
Requires: | ||
Ensures: | ||
1: | Multiband Polarimetric Image Dataset Analysis | |
2: | Calculate Spectral Reflectance of the Mean Foreground (FG) and BG Area | |
3: | Calculate Correlation Among Bands and Polarimetric Orientation | |
4: | Calculate First Order and Second Order Texture Features | |
5: | ifInformation Differs Significantly among Bands, then | |
6: | Polarimetric Orientation-wise Multiband Fusion (Algorithm 2) | |
7: | Evaluate and Performance of the Fusion Method Statistically | |
8: | end if | |
9: | if, , , exist then | |
10: | Compute Stokes Vector: | |
11: | Compute Polarimetric Components:, and | |
12: | end if | |
13: | BS in Polarimetric Imagery (Algorithm 3) | |
14: | Evaluate and Compare Performance of the Proposed Method Statistically |
4.2. Hybrid Fusion Framework and Algorithm
Algorithm 2. Polarimetric Orientation-wise Multiband Fusion | |||
Requires: | |||
Ensures: | |||
1: | forall polarimetricdo | ||
2: | foralldo | ||
3: | Create a Gaussian Low Pass Filter () and Gaussian High Pass Filter () | ||
4: | Calculate a Discreate Fourier Transform | ||
5: | Multiplywithand | ||
6: | Convert the Result to the Spatial Domain by inverting Multiplication result Apply Inverse Fast Fourier Transformation to the multiplication results which produce band-wise final results asand | ||
7: | end for | ||
8: | Calculate the covariance and eigenvector ofand | ||
9: | Calculate the first Principal Component ofand | ||
10: | Calculate the fused imagery by adding both Principal Components | ||
11: | end for |
4.3. Calculation of Polarimetric Components
4.4. Proposed BS Algorithm
Algorithm 3. BS in Polarimetric Imagery | |||
Requires: | |||
Ensures: | |||
1: | if,,,exist then | ||
2: | Significant Foreground Mask Generation | ||
3: | Construct an intensity invariant mask through differentiating the median filtering version of unpolarized and polarized imagery | ||
4: | Calculate a strongly unpolarized foreground mask in two different ways utilizingand | ||
5: | Calculate a strongly polarized foreground mask | ||
6: | Calculate a strong light intensity mask based on azimuth angle and. | ||
7: | Combine steps 3-6 and apply a morphological operation to segment the total background area of a scene. | ||
8: | end if |
5. Experimental Results
5.1. Performance Evaluation of MSPI Fusion
5.1.1. Selection of Fusion Metric
5.1.2. Observation of Fusion Quality
5.1.3. Comparison of Performances of Fusion Methods
5.1.4. Visualization of Fusion Performance
5.2. Calculation of Polarimetric Component
5.3. Performance Evaluation of MSPI BS
5.3.1. Selection of BS Metric
5.3.2. Generation of Ground Truth
5.3.3. Comparison of BS Accuracy: Direct vs. Fusion
5.3.4. Visualization of Performance of BS
5.3.5. Comparison of BS Accuracy: Proposed Method vs. Those in Literature
5.4. Computational Time Analysis
6. Conclusions
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
References
- Lapray, P.J.; Gendre, L.; Foulonneau, A.; Bigué, L. Database of polarimetric and multispectral images in the visible and NIR regions. In Unconventional Optical Imaging, Proceedings of the SPIE, Strasbourg, France, 24 May 2018; SPIE: Bellingham, WA, USA, 2018. [Google Scholar]
- Yan, Q.; Shen, X.; Xu, L.; Zhuo, S.; Zhang, X.; Shen, L.; Jia, J. Crossfield joint image restoration via scale map. In Proceedings of the IEEE International Conference on Computer Vision, Sydney, NSW, Australia, 1–8 December 2013. [Google Scholar]
- Schaul, L.; Fredembach, C.; Susstrunk, S. Color image dehazing using the near-infrared. In Proceedings of the 16th IEEE International Conference on Image Processing (ICIP), Chiang Mai, Thailand, 7 November 2009. [Google Scholar]
- Salamati, N.; Larlus, D.; Csurka, G.; Süsstrunk, S. Semantic image segmentation using visible and near-infrared channels. In Proceedings of the European Conference on Computer Vision, Florence, Italy, 7–13 October 2012. [Google Scholar]
- Berns, R.S.; Imai, F.H.; Burns, P.D.; Tzeng, D.Y. Multispectral-based color reproduction research at the Munsell Color Science Laboratory. In Electronic Imaging: Processing, Printing, and Publishing in Color, Proceedings of the SPIE, Zurich, Switzerland, 7 September 1998; SPIE: Bellingham, WA, USA, 1998. [Google Scholar]
- Thomas, J.B. Illuminant estimation from uncalibrated multispectral images. In Proceedings of the 2015 Colour and Visual Computing Symposium (CVCS), Gjovik, Norway, 25–26 August 2015. [Google Scholar]
- Motohka, T.; Nasahara, K.N.; Oguma, H.; Tsuchida, S. Applicability of green-red vegetation index for remote sensing of vegetation phenology. Remote Sens. 2010, 2, 2369–2387. [Google Scholar]
- Dandois, J.P.; Ellis, E.C. Remote sensing of vegetation structure using computer vision. Remote. Sens. 2010, 2, 1157–1176. [Google Scholar] [CrossRef] [Green Version]
- Rfenacht, D.; Fredembach, C.; Süsstrunk, S. Automatic and accurate shadow detection using near-infrared information. IEEE Trans. Pattern Anal. Mach. Intell. 2014, 36, 1672–1678. [Google Scholar] [CrossRef] [PubMed]
- Sobral, A.; Javed, S.; Ki Jung, S.; Bouwmans, T.; Zahzah, E.H. Online stochastic tensor decomposition for background subtraction in multispectral video sequences. In Proceedings of the 2015 IEEE International Conference on Computer Vision Workshop (ICCVW), Santiago, Chile, 7–13 December 2015. [Google Scholar]
- Tyo, J.S.; Goldstein, D.L.; Chenault, D.B.; Shaw, J.A. Review of passive imaging polarimetry for remote sensing applications. Appl. Opt. 2006, 45, 5453–5469. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Nayar, S.K.; Fang, X.-S.; Boult, T. Separation of reflection components using color and polarization. Int. J. Comput. Vis. 1997, 21, 163–186. [Google Scholar] [CrossRef]
- Wolff, L.B. Polarization-based material classification from specular reflection. IEEE Trans. Pattern Anal. Mach. Intell. 1990, 12, 1059–1071. [Google Scholar] [CrossRef]
- Atkinson, G.A.; Hancock, E.R. Shape estimation using polarization and shading from two views. IEEE Trans. Pattern Anal. Mach. Intell. 2007, 29, 2001–2017. [Google Scholar]
- Tan, J.; Zhang, J.; Zhang, Y. Target detection for polarized hyperspectral images based on tensor decomposition. IEEE Geosci. Remote Sens. Lett. 2017, 14, 674–678. [Google Scholar] [CrossRef]
- Goudail, F.; Terrier, P.; Takakura, Y.; Bigue, L.; Galland, F.; DeVlaminck, V. Target detection with a liquid-crystal-based passive stokes polarimeter. Appl. Opt. 2004, 43, 274–282. [Google Scholar] [CrossRef] [Green Version]
- Denes, L.J.; Gottlieb, M.S.; Kaminsky, B.; Huber, D.F. Spectropolarimetric imaging for object recognition. In Proceedings of the 26th AIPR Workshop: Exploiting New Image Sources and Sensors, Washington, DC, USA, 1 March 1998. [Google Scholar]
- Romano, J.M.; Rosario, D.; McCarthy, J. Day/night polarimetric anomaly detection using SPICE imagery. IEEE Trans. Geosci. Remote Sens. 2012, 50, 5014–5023. [Google Scholar] [CrossRef]
- Islam, M.N.; Tahtali, M.; Pickering, M. Man-made object separation using polarimetric imagery. In Proceedings of the SPIE Future Sensing Technologies, Tokyo, Japan, 12–14 November 2019. [Google Scholar]
- Zhou, P.C.; Liu, C.C. Camouflaged target separation by spectral-polarimetric imagery fusion with shearlet transform and clustering segmentation. In Proceedings of the International Symposium on Photoelectronic Detection and Imaging 2013: Imaging Sensors and Applications, Bei**g, China, 21 August 2013. [Google Scholar]
- Domadiya, P.; Shah, P.; Mitra, S.K. Fast and Accurate Foreground Background Separation for Video Surveillance. In Proceedings of the 6th International Conference on Pattern Recognition and Machine Intelligence (PReMI), Warsaw, Poland, 30 June–3 July 2015; Springer: Cham, Switzerland, 2015. [Google Scholar]
- Bouwmans, T. Traditional and recent approaches in background modeling for foreground detection: An overview. Comput. Sci. Rev. 2014, 11, 31–66. [Google Scholar] [CrossRef]
- Perazzi, F.; Pont-Tuset, J.; McWilliams, B.; Van Gool, L.; Gross, M.; Sorkine-Hornung, A. A benchmark dataset and evaluation methodology for video object segmentation. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016. [Google Scholar]
- Zitova, B.; Flusser, J. Image registration methods: A survey. Image Vis. Comput. 2003, 21, 977–1000. [Google Scholar] [CrossRef] [Green Version]
- Benezeth, Y.; Sidibé, D.; Thomas, J.B. Background subtraction with multispectral video sequences. In Proceedings of the IEEE International Conference on Robotics and Automation workshop on Nonclassical Cameras, Camera Networks and Omnidirectional Vision (OMNIVIS), Hong Kong, China, 11 June 2014. HAL-00986168f. [Google Scholar]
- Ma, J.; Ma, Y.; Li, C. Infrared and visible image fusion methods and applications: A survey. Inf. Fusion 2019, 45, 153–178. [Google Scholar] [CrossRef]
- Zhan, L.; Zhuang, Y.; Huang, L. Infrared and visible images fusion method based on discrete wavelet transform. J. Comput. 2017, 28, 57–71. [Google Scholar] [CrossRef]
- Li, H.; Liu, L.; Huang, W.; Yue, C. An improved fusion algorithm for infrared and visible images based on multi-scale transform. Infrared Phys. Technol. 2016, 74, 28–37. [Google Scholar] [CrossRef]
- Zhao, Y.Q.; Zhang, L.; Zhang, D.; Pan, Q. Object separation by polarimetric and spectral imagery fusion. Comput. Vis. Image Underst. 2009, 113, 855–866. [Google Scholar] [CrossRef]
- Weinberger, M.J.; Seroussi, G.; Sapiro, G. The LOCO-I lossless image compression algorithm: Principles and standardization into JPEG-LS. IEEE Trans. Image Process. 2000, 12, 1309–1324. [Google Scholar] [CrossRef] [Green Version]
- Rizzo, F.; Carpentieri, B.; Motta, G.; Storer, J.A. Low-Complexity Lossless Compression of Hyperspectral Imagery via Linear Prediction. IEEE Signal Process. Lett. 2005, 12, 138–141. [Google Scholar] [CrossRef]
- Seki, M.; Wada, T.; Fujiwara, H.; Sumi, K. Background subtraction based on cooccurrence of image variations. In Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Madison, WI, USA, 18–20 June 2003. [Google Scholar]
- Naidu, V.P.S.; Raol, J.R. Pixel-level image fusion using wavelets and principal component analysis. Def. Sci. J. 2008, 58, 338. [Google Scholar] [CrossRef]
- Bavirisetti, D.P.; **, J. Remote Sensing Digital Image Analysis: An Introduction, 4th ed.; Springer: Berlin/Heidelberg, Germany, 1999. [Google Scholar]
- Haralick, R.M.; Shanmugam, K.; Dinstein, I.H. Textural features for image classification. IEEE Trans. Syst. Man Cybern 1973, SMC-3, 610–621. [Google Scholar] [CrossRef] [Green Version]
- Zhang, X.; Cui, J.; Wang, W.; Lin, C. A study for texture feature extraction of high-resolution satellite images based on a direction measure and gray level co-occurrence matrix fusion algorithm. Sensors 2017, 17, 1474. [Google Scholar] [CrossRef] [Green Version]
- Stokes, G.G. On the composition and resolution of streams of polarized light from different sources. Trans. Camb. Philos. Soc. 1851, 9, 399. [Google Scholar]
- Shannon, C.E. A mathematical theory of communication. Bell Syst. Tech. J. 1948, 27, 379–423. [Google Scholar] [CrossRef] [Green Version]
- Somvanshi, S.S.; Kunwar, P.; Tomar, S.; Singh, M. Comparative statistical analysis of the quality of image enhancement techniques. Int. J. Image Data Fusion 2017, 9, 131–151. [Google Scholar] [CrossRef]
- Haghighat, M.B.A.; Aghagolzadeh, A.; Seyedarabi, H. A non-reference image fusion metric based on mutual information of image features. Comput. Electr. Eng. 2011, 37, 744–756. [Google Scholar] [CrossRef]
- Rani, K.; Sharma, R. Study of different image fusion algorithm. Int. J. Emerg. Technol. Adv. Eng. 2013, 3, 288–291. [Google Scholar]
- Chiu, S.Y.; Chiu, C.C.; Xu, S.S.D. A Background Subtraction Algorithm in Complex Environments Based on Category Entropy Analysis. Appl. Sci. 2018, 8, 885. [Google Scholar] [CrossRef] [Green Version]
MAPE | PSNR | PCOR | MI | ||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0° | 45° | 90° | 135° | 0° | 45° | 90° | 135° | 0° | 45° | 90° | 135° | 0° | 45° | 90° | 135° | ||
Liquid Scene | DWT [37] | 26.28 | 33.42 | 41.25 | 31.74 | 31.94 | 32.96 | 31.73 | 32.28 | 0.99 | 0.99 | 0.99 | 0.98 | 2.32 | 2.42 | 2.32 | 2.18 |
PCA [33] | 19.23 | 26.47 | 34.69 | 22.54 | 29.05 | 28.88 | 28.86 | 28.86 | 0.99 | 0.99 | 0.99 | 0.99 | 2.62 | 2.72 | 2.64 | 2.49 | |
DCT–LP [40] | 146.25 | 167.14 | 189.36 | 150.89 | 23.19 | 24.57 | 24.12 | 22.99 | 0.81 | 0.83 | 0.84 | 0.82 | 0.93 | 1.09 | 1.01 | 0.92 | |
SHT [20] | 93.80 | 123.43 | 164.35 | 116.91 | 19.60 | 19.32 | 20.01 | 19.46 | 0.86 | 0.88 | 0.88 | 0.86 | 1.32 | 1.48 | 1.37 | 1.25 | |
FPDE [34] | 16.79 | 21.53 | 27.83 | 19.28 | 30.48 | 31.08 | 31.24 | 30.51 | 0.99 | 0.99 | 0.99 | 0.99 | 2.73 | 2.87 | 2.77 | 2.63 | |
Proposed | 30.86 | 39.55 | 52.80 | 35.23 | 18.90 | 19.37 | 19.49 | 19.10 | 0.99 | 0.99 | 0.99 | 0.99 | 2.89 | 2.97 | 2.84 | 2.75 | |
Food Scene | DWT [37] | 11.13 | 16.33 | 15.08 | 14.22 | 32.56 | 32.28 | 31.37 | 32.04 | 0.99 | 0.99 | 0.99 | 0.99 | 2.68 | 2.60 | 2.57 | 2.76 |
PCA [33] | 5.06 | 8.81 | 8.56 | 6.64 | 30.19 | 28.13 | 27.53 | 30.00 | 0.99 | 0.99 | 0.98 | 0.99 | 3.02 | 2.76 | 2.68 | 3.00 | |
DCT–LP [40] | 93.21 | 156.36 | 157.00 | 122.33 | 21.64 | 21.59 | 21.24 | 22.05 | 0.84 | 0.81 | 0.81 | 0.83 | 1.17 | 1.04 | 1.02 | 1.16 | |
SHT [20] | 20.39 | 30.40 | 30.56 | 22.83 | 20.47 | 19.21 | 18.38 | 20.01 | 0.95 | 0.92 | 0.92 | 0.94 | 1.87 | 1.70 | 1.62 | 1.85 | |
FPDE [34] | 19.79 | 31.96 | 30.23 | 25.02 | 20.11 | 18.76 | 19.16 | 19.93 | 0.90 | 0.80 | 0.81 | 0.89 | 2.21 | 1.77 | 1.77 | 2.20 | |
Proposed | 10.14 | 15.66 | 15.08 | 12.22 | 16.90 | 18.15 | 18.32 | 17.14 | 1.00 | 0.99 | 0.99 | 1.00 | 3.32 | 3.07 | 3.05 | 3.43 | |
Leaf Scene | DWT [37] | 15.00 | 14.45 | 14.52 | 14.95 | 26.14 | 26.63 | 26.60 | 26.29 | 0.98 | 0.97 | 0.97 | 0.98 | 2.20 | 2.07 | 2.07 | 2.22 |
PCA [33] | 5.32 | 8.64 | 9.51 | 5.37 | 26.95 | 22.53 | 21.97 | 26.60 | 0.99 | 0.94 | 0.93 | 0.99 | 2.78 | 2.35 | 2.30 | 2.75 | |
DCT–LP [40] | 55.15 | 63.95 | 62.89 | 54.54 | 14.87 | 14.53 | 14.61 | 14.82 | 0.84 | 0.76 | 0.75 | 0.84 | 1.17 | 1.10 | 1.12 | 1.18 | |
SHT [20] | 26.72 | 32.41 | 36.56 | 26.46 | 13.19 | 11.96 | 11.89 | 13.04 | 0.86 | 0.76 | 0.74 | 0.85 | 1.58 | 1.57 | 1.56 | 1.60 | |
FPDE [34] | 9.51 | 24.55 | 32.66 | 10.82 | 25.41 | 20.43 | 18.84 | 24.64 | 0.98 | 0.89 | 0.86 | 0.97 | 3.43 | 2.01 | 1.48 | 3.28 | |
Proposed | 10.41 | 14.39 | 15.58 | 10.30 | 10.54 | 12.14 | 12.22 | 10.75 | 0.99 | 0.96 | 0.95 | 0.99 | 2.89 | 2.52 | 2.47 | 2.88 | |
Average | DWT [37] | 17.47 | 21.40 | 23.62 | 20.30 | 30.21 | 30.63 | 29.90 | 30.20 | 0.99 | 0.98 | 0.98 | 0.99 | 2.40 | 2.36 | 2.32 | 2.39 |
PCA [33] | 9.87 | 14.64 | 17.59 | 11.52 | 28.73 | 26.51 | 26.12 | 28.49 | 0.99 | 0.97 | 0.97 | 0.99 | 2.81 | 2.61 | 2.54 | 2.74 | |
DCT–LP [40] | 98.20 | 129.15 | 136.42 | 109.25 | 19.90 | 20.23 | 19.99 | 19.95 | 0.83 | 0.80 | 0.80 | 0.83 | 1.09 | 1.08 | 1.05 | 1.09 | |
SHT [20] | 46.97 | 62.08 | 77.16 | 55.40 | 17.75 | 16.83 | 16.76 | 17.50 | 0.89 | 0.85 | 0.85 | 0.88 | 1.59 | 1.58 | 1.52 | 1.57 | |
FPDE [34] | 15.36 | 26.01 | 30.24 | 18.37 | 25.33 | 23.42 | 23.08 | 25.03 | 0.95 | 0.89 | 0.88 | 0.95 | 2.79 | 2.22 | 2.01 | 2.70 | |
Proposed | 17.14 | 23.20 | 27.82 | 19.25 | 15.44 | 16.55 | 16.68 | 15.66 | 0.99 | 0.98 | 0.98 | 0.99 | 3.03 | 2.86 | 2.79 | 3.02 |
Direct BG Separation | Fusion-based BG Segmentation | |||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
AC | SP | SN | GM | PR | RC | F1S | AC | SP | SN | GM | PR | RC | F1S | |||
Liquid Scene | B-1 | 0.91 | 0.68 | 0.96 | 0.80 | 0.93 | 0.96 | 0.94 | DWT [37] | 0.96 | 0.97 | 0.96 | 0.97 | 0.99 | 0.96 | 0.98 |
B-2 | 0.91 | 0.74 | 0.95 | 0.84 | 0.94 | 0.95 | 0.95 | PCA [33] | 0.96 | 0.95 | 0.96 | 0.95 | 0.99 | 0.96 | 0.97 | |
B-3 | 0.92 | 0.84 | 0.94 | 0.89 | 0.96 | 0.94 | 0.95 | AVG [36] | 0.96 | 0.98 | 0.95 | 0.97 | 1.00 | 0.95 | 0.97 | |
B-4 | 0.91 | 0.61 | 0.98 | 0.77 | 0.92 | 0.98 | 0.95 | DCT–LP [40] | 0.94 | 0.73 | 0.98 | 0.85 | 0.94 | 0.98 | 0.96 | |
B-5 | 0.91 | 0.78 | 0.94 | 0.86 | 0.95 | 0.94 | 0.94 | SHT [20] | 0.95 | 0.81 | 0.98 | 0.89 | 0.96 | 0.98 | 0.97 | |
B-6 | 0.95 | 0.81 | 0.98 | 0.89 | 0.96 | 0.98 | 0.97 | FPDE [34] | 0.92 | 0.98 | 0.91 | 0.94 | 1.00 | 0.91 | 0.95 | |
Proposed | 0.97 | 0.98 | 0.97 | 0.97 | 1.00 | 0.97 | 0.98 | |||||||||
Food Scene | B-1 | 0.62 | 0.69 | 0.59 | 0.64 | 0.81 | 0.59 | 0.68 | DWT [37] | 0.91 | 0.85 | 0.93 | 0.89 | 0.93 | 0.93 | 0.93 |
B-2 | 0.59 | 0.76 | 0.51 | 0.63 | 0.83 | 0.51 | 0.63 | PCA [33] | 0.95 | 0.85 | 0.99 | 0.92 | 0.94 | 0.99 | 0.96 | |
B-3 | 0.54 | 0.88 | 0.39 | 0.59 | 0.88 | 0.39 | 0.54 | AVG [36] | 0.92 | 0.87 | 0.94 | 0.90 | 0.94 | 0.94 | 0.94 | |
B-4 | 0.60 | 0.67 | 0.56 | 0.61 | 0.79 | 0.56 | 0.66 | DCT–LP [40] | 0.92 | 0.75 | 0.99 | 0.86 | 0.90 | 0.99 | 0.94 | |
B-5 | 0.53 | 0.87 | 0.37 | 0.57 | 0.86 | 0.37 | 0.52 | SHT [20] | 0.85 | 0.73 | 0.91 | 0.82 | 0.88 | 0.91 | 0.89 | |
B-6 | 0.93 | 0.80 | 0.99 | 0.89 | 0.92 | 0.99 | 0.95 | FPDE [34] | 0.84 | 0.79 | 0.86 | 0.82 | 0.90 | 0.86 | 0.88 | |
Proposed | 0.96 | 0.90 | 0.99 | 0.94 | 0.96 | 0.99 | 0.97 | |||||||||
Leaf Scene | B-1 | 0.65 | 0.25 | 0.77 | 0.44 | 0.77 | 0.77 | 0.77 | DWT [37] | 0.71 | 0.90 | 0.65 | 0.76 | 0.95 | 0.65 | 0.77 |
B-2 | 0.63 | 0.29 | 0.73 | 0.46 | 0.77 | 0.73 | 0.75 | PCA [33] | 0.80 | 0.82 | 0.79 | 0.80 | 0.93 | 0.79 | 0.86 | |
B-3 | 0.67 | 0.63 | 0.68 | 0.66 | 0.86 | 0.68 | 0.76 | AVG [36] | 0.81 | 0.80 | 0.82 | 0.81 | 0.93 | 0.82 | 0.87 | |
B-4 | 0.68 | 0.24 | 0.81 | 0.44 | 0.78 | 0.81 | 0.79 | DCT–LP [40] | 0.92 | 0.81 | 0.95 | 0.88 | 0.94 | 0.95 | 0.94 | |
B-5 | 0.55 | 0.52 | 0.57 | 0.54 | 0.79 | 0.57 | 0.66 | SHT [20] | 0.76 | 0.55 | 0.82 | 0.67 | 0.86 | 0.82 | 0.84 | |
B-6 | 0.82 | 0.26 | 1.00 | 0.50 | 0.81 | 1.00 | 0.89 | FPDE [34] | 0.58 | 0.83 | 0.50 | 0.65 | 0.91 | 0.50 | 0.65 | |
Proposed | 0.89 | 0.84 | 0.90 | 0.87 | 0.95 | 0.90 | 0.92 | |||||||||
Average | Direct | 0.74 | 0.63 | 0.76 | 0.67 | 0.86 | 0.76 | 0.80 | Fusion | 0.88 | 0.84 | 0.89 | 0.86 | 0.94 | 0.89 | 0.91 |
Liquid Scene | Food Scene | Leaf Scene | |||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
AC | SP | SN | GM | PR | RC | F1S | AC | SP | SN | GM | PR | RC | F1S | AC | SP | SN | GM | PR | RC | F1S | |
Wolf [13] | 0.69 | 0.00 | 0.84 | 0.06 | 0.79 | 0.84 | 0.82 | 0.57 | 0.99 | 0.38 | 0.61 | 0.99 | 0.38 | 0.55 | 0.65 | 0.40 | 0.73 | 0.54 | 0.80 | 0.73 | 0.76 |
Zhao [29] | 0.95 | 1.00 | 0.94 | 0.97 | 1.00 | 0.94 | 0.97 | 0.81 | 0.38 | 1.00 | 0.62 | 0.78 | 1.00 | 0.88 | 0.44 | 0.93 | 0.28 | 0.51 | 0.93 | 0.28 | 0.44 |
Zhou [20] | 0.67 | 1.00 | 0.60 | 0.78 | 1.00 | 0.60 | 0.75 | 0.70 | 0.85 | 0.62 | 0.73 | 0.90 | 0.62 | 0.74 | 0.58 | 0.40 | 0.64 | 0.51 | 0.78 | 0.64 | 0.70 |
Lu [59] | 0.75 | 1.00 | 0.69 | 0.83 | 1.00 | 0.69 | 0.82 | 0.89 | 0.72 | 0.96 | 0.83 | 0.88 | 0.96 | 0.92 | 0.51 | 0.62 | 0.48 | 0.54 | 0.80 | 0.48 | 0.60 |
Proposed | 0.97 | 0.98 | 0.97 | 0.97 | 1.00 | 0.97 | 0.98 | 0.96 | 0.90 | 0.99 | 0.94 | 0.96 | 0.99 | 0.97 | 0.89 | 0.84 | 0.90 | 0.87 | 0.95 | 0.90 | 0.92 |
Running Time (Seconds) – Multiband Fusion | Running Time (Seconds) – Polarimetric BS | ||||||||
---|---|---|---|---|---|---|---|---|---|
Liquid | Food | Leaf | Average | Liquid | Food | Leaf | Average | ||
DWT [37] | 55.76 | 57.01 | 57.51 | 56.76 | Wolf [13] | 10.74 | 10.14 | 11.35 | 10.74 |
PCA [33] | 38.70 | 38.19 | 37.26 | 38.05 | Zhao [29] | 8.66 | 8.93 | 7.40 | 8.33 |
AVG [36] | 63.05 | 63.47 | 62.94 | 63.15 | Zhou [20] | 12.38 | 8.95 | 12.57 | 11.30 |
DCT–LP [40] | 56.40 | 56.00 | 56.70 | 56.37 | Lu [59] | 10.81 | 10.61 | 11.60 | 11.01 |
SHT [20] | 162.06 | 163.48 | 160.23 | 161.92 | Proposed | 2.52 | 3.45 | 3.74 | 3.23 |
FPDE [34] | 113.20 | 113.91 | 113.90 | 113.67 | |||||
Proposed | 44.65 | 44.62 | 44.58 | 44.62 |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Islam, M.N.; Tahtali, M.; Pickering, M. Hybrid Fusion-Based Background Segmentation in Multispectral Polarimetric Imagery. Remote Sens. 2020, 12, 1776. https://doi.org/10.3390/rs12111776
Islam MN, Tahtali M, Pickering M. Hybrid Fusion-Based Background Segmentation in Multispectral Polarimetric Imagery. Remote Sensing. 2020; 12(11):1776. https://doi.org/10.3390/rs12111776
Chicago/Turabian StyleIslam, Md Nazrul, Murat Tahtali, and Mark Pickering. 2020. "Hybrid Fusion-Based Background Segmentation in Multispectral Polarimetric Imagery" Remote Sensing 12, no. 11: 1776. https://doi.org/10.3390/rs12111776