Class-Wise Feature Alignment Based Transfer Network for Multi-Temporal Remote Sensing Image Classification
-
摘要: 为了在目标域遥感图像不存在标记数据的情况下实现自动分类，论文提出一种基于特征对齐的迁移网络.网络以各类类心对齐和协方差对齐作为迁移策略，全面描述域间各类别之间的对应关系，实现知识迁移.另外，网络采用线性修正单元作为激活函数，能够产生稀疏特征，提高分类效果.该迁移网络能够同时获得对齐的特征和自适应分类器，不需要目标域的标记数据，实现无监督迁移学习.在多时相的Hyperion高光谱遥感图像和WorldView-2多光谱遥感图像上的实验结果证明了该迁移网络的有效性.Abstract: A transfer neural network based on feature alignment is proposed for classification of multi-temporal remote sensing image. In the network, the mean vector and covariance matrix of the sample data of each class are used to describe the data distribution, and the domain shift in terms of the first and second statistics can be reduced. In addition, rectified linear unit is utilized as the activation function, which can produce sparse features and improve the classification performance. In the transfer neural network, both aligned features and adaptive classifiers can be obtained simultaneously and unsupervised domain adaptation is achieved when there is no labeled data in the target image. The experimental results of multi-temporal Hyperion hyperspectral remote sensing images and WorldView-2 multispectral remote sensing images demonstrate the effectiveness of the proposed transfer neural network.
-
Key words:
- transfer neural network /
- classification /
- class-wise feature alignment /
- remote sensing
-
图 3 博茨瓦纳共和国奥卡万戈三角洲的Hyperion多时相遥感图像
a. 5月图像；b. 5月标记图；c. 6月图像；d. 6月标记图；e. 7月图像；f. 7月标记图
Fig. 3. Multi-temporal Hyperion images captured from Okavango Delta, Botswana
图 4 湖北武汉的WorldView-2多时相遥感图像
a.2011年图像；b.2011年标记图；c.2012年图像；d.2012年标记图
Fig. 4. Multi-temporal WordView-2 images captured from Wuhan, Hubei Province, China
图 5 BOT5-6数据的CFATN对齐效果
a.第1类对齐前；b.第1类对齐后；c.第2类对齐前；d.第2类对齐后；e.第6类对齐前；f.第6类对齐后；g.第9类对齐前；h.第9类对齐后
Fig. 5. Alignment performance of CFATN for "May-June" data of Hyperion-BOT image
表 1 Hyperion-BOT和WV2-WH多时相遥感图像的类别名称和样本数目
Table 1. The category and sample number of the multi- temporal images captured by different sensors
ID 类名 5月 6月 7月 Hyperion-BOT 1 水体 297 361 185 2 泛滥平原 437 308 96 3 河岸 448 303 164 4 火迹 354 335 186 5 岛屿内陆 337 370 131 6 林地 357 324 169 7 稀树草原 330 342 171 8 短可乐豆木 239 299 152 9 裸露土地 215 229 96 ID 类名 2011年 2012年 WV2-WH 1 红房顶 2 511 2 963 2 森林 3 592 3 144 3 灰房顶 4 425 4 528 4 白房顶 3 082 5 301 
下载: 导出CSV
表 2 不同迁移学习算法的多时相遥感图像的整体分类精度(%)和Kappa系数
Table 2. OA(%) and Kappa coefficient of multi-temporal images based on different transfer learning algorithms
数据集 SVM RNN SSTCA DTN TLDA DANN CFATN 精度(%) Hyperion-BOT BOT5-6 69.63 74.16 76.91 85.51 78.44 88.23 93.17 BOT6-5 60.62 68.38 77.27 79.36 69.21 78.15 84.01 BOT5-7 88.05 75.56 88.81 87.70 84.15 86.79 91.56 BOT7-5 56.44 60.58 68.48 79.99 67.85 68.69 84.47 BOT6-7 90.59 91.48 91.63 91.26 92.30 94.47 95.70 BOT7-6 88.05 85.44 87.53 91.68 90.39 89.40 94.85 WV2-WH 2011—2012 84.92 84.58 89.07 89.58 90.17 93.73 94.42 2012—2011 69.67 85.86 85.66 78.33 85.94 87.57 93.11 Kappa系数 Hyperion-BOT BOT5-6 0.66 0.71 0.74 0.84 0.77 0.87 0.92 BOT6-5 0.56 0.64 0.74 0.77 0.65 0.75 0.82 BOT5-7 0.87 0.72 0.87 0.86 0.82 0.85 0.90 BOT7-5 0.51 0.56 0.65 0.77 0.64 0.65 0.82 BOT6-7 0.89 0.90 0.91 0.90 0.91 0.94 0.95 BOT7-6 0.87 0.84 0.86 0.91 0.89 0.88 0.94 WV2-WH 2011—2012 0.80 0.79 0.85 0.86 0.87 0.91 0.92 2012—2011 0.58 0.81 0.81 0.70 0.81 0.83 0.90
下载: 导出CSV
表 3 不同激活函数CFATN算法的多时相遥感图像整体分类精度(%)
Table 3. OA(%) of multi-temporal image based on the CFATN algorithm with different activation functions
激活函数 Hyperion-BOT WV2-WH BOT5-6 BOT6-5 BOT5-7 BOT7-5 BOT6-7 BOT7-6 2011—2012 2012—2011 ReLU 93.17 84.01 91.56 84.85 95.70 94.85 94.21 93.11 Sigmoid 90.70 83.78 89.63 79.50 95.26 91.57 92.55 91.20
下载: 导出CSV
-
[1] Bruzzone, L., Marconcini, M., 2010. Domain Adaptation Problems: A DASVM Classification Technique and a Circular Validation Strategy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(5): 770-787. https://doi.org/10.1109/TPAMI.2009.57 [2] Cheng, G.X., Niu, R.Q., Zhang, K.X., et al., 2018. Opencast Mining Area Recognition in High-Resolution Remote Sensing Images Using Convolutional Neural Networks. Earth Science, 43(Suppl. 2): 256-262(in Chinese with English abstract). http://en.cnki.com.cn/Article_en/CJFDTotal-DQKX2018S2021.htm [3] Deng, B., Jia, S., Shi, D. M., 2020. Deep Metric Learning-Based Feature Embedding for Hyperspectral Image Classification. IEEE Transactions on Geoscience and Remote Sensing, 58(2): 1422-1435. https://doi.org/10.1109/TGRS.2019.2946318 [4] Ganin, Y., Ustinova, E., Ajakan, H., et al., 2017. Domain-Adversarial Training of Neural Networks. Domain Adaptation in Computer Vision Applications. Springer International Publishing, Cham, 189-209. https://doi.org/10.1007/978-3-319-58347-1_10 [5] Gao, C.X., Sang, N., 2014. Deep Learning for Object Detection in Remote Sensing Image. Bulletin of Surveying and Mapping, (S1): 108-111(in Chinese with English abstract). http://search.cnki.net/down/default.aspx?filename=CHTB2014S1025&dbcode=CJFD&year=2014&dflag=pdfdown [6] Glorot, X., Bordes, A., Bengio, Y., 2011. Deep Sparse Rectifier Neural Networks. In: Gordon, G. J., Dunson, D. B., Dudík, M., eds., Proceedings of International Conference on 14th Artificial Intelligence and Statistics, AISTATS 2011. JMLR, Cambridge, 315-323. [7] Jhuo, I. H., Liu, D., Lee, D.T., et al., 2012. Robust Visual Domain Adaptation with Low-Rank Reconstruction. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2012. IEEE Computer Society, Los Alamitos, 2168-2175. https://doi.org/10.1109/CVPR.2012.6247924 [8] Li, W.K., Zhang, W., Qin, J.H., et al., 2020. "Expansion-Fusion" Extraction of Surface Gully Area Based on DEM and High-Resolution Remote Sensing Images. Earth Science, 45(6): 1948-1955(in Chinese with English abstract). [9] Li, Z. K., Tang, X. Y., Li, W., et al., 2020. A Two-Stage Deep Domain Adaptation Method for Hyperspectral Image Classification. Remote Sensing, 12(7): 1054-1073. https://doi.org/10.3390/rs12071054 [10] Liu, D.W., Han, L., Han, X.Y., 2016. High Spatial Resolution Remote Sensing Image Classification Based on Deep Learning. Acta Optica Sinica, 36(4): 0428001(in Chinese with English abstract). doi: 10.3788/AOS201636.0428001 [11] Liu, Z.X., Ma, L., Du, Q., 2020. Class-Wise Distribution Adaptation for Unsupervised Classification of Hyperspectral Remote Sensing Images. IEEE Transactions on Geoscience and Remote Sensing, 59(1): 508-521. https://doi.org/10.1109/TGRS.2020.2997863 [12] Ma, L., Crawford, M. M., Zhu, L., et al., 2019. Centroid and Covariance Alignment-Based Domain Adaptation for Unsupervised Classification of Remote Sensing Images. IEEE Transactions on Geoscience and Remote Sensing, 57(4): 2305-2323. https://doi.org/10.1109/TGRS.2018.2872850 [13] Matasci, G., Volpi, M., Kanevski, M., et al., 2015. Semisupervised Transfer Component Analysis for Domain Adaptation in Remote Sensing Image Classification. IEEE Transactions on Geoscience and Remote Sensing, 53(7): 3550-3564. doi: 10.1109/TGRS.2014.2377785 [14] Pan, S. J., Yang, Q., 2010. A Survey on Transfer Learning. IEEE Transactions on Knowledge and Data Engineering, 22(10): 1345-1359. https://doi.org/10.1109/TKDE.2009.191 [15] Rajan, S., Ghosh, J., Crawford, M.M., 2006. Exploiting Class Hierarchies for Knowledge Transfer in Hyperspectral Data. IEEE Transactions on Geoscience and Remote Sensing, 44(11): 3408-3417. https://doi.org/10.1109/TGRS.2006.878442 [16] Sun, H., Liu, S., Zhou, S. L., et al., 2016. Transfer Sparse Subspace Analysis for Unsupervised Cross-View Scene Model Adaptation. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 9(7): 2901-2909. https://doi.org/10.1109/JSTARS.2015.2500961 [17] Sun, Z., Wang, C., Wang, H. Y., et al., 2013. Learn Multiple-Kernel SVMS for Domain Adaptation in Hyperspectral Data. IEEE Geoscience and Remote Sensing Letters, 10(5): 1224-1228. https://doi.org/10.1109/LGRS.2012.2236818 [18] Sun, B. C., Feng, J. S., Saenko, K., 2017. Correlation Alignment for Unsupervised Domain Adaptation. In: Advances in Computer Vision and Pattern Recognition, Springer, Berlin, 153-171. https://doi.org/10.1007/978-3-319-58347-1_8 [19] Tong, X. Y., Xia, G. S., Lu, Q. K., et al., 2020. Land-Cover Classification with High-Resolution Remote Sensing Images Using Transferable Deep Models. Remote Sensing of Environment, 237: 111322. https://doi.org/10.1016/j.rse.2019.111322 [20] Vakalopoulou, M., Karantzalos, K., Komodakis, N., et al., 2015. Building Detection in Very High Resolution Multispectral Data with Deep Learning Features. In: Proceedings of IEEE International Geoscience and Remote Sensing Symposium (IGARSS). IEEE, Milan, Italy, 1873-1876. https://doi.org/10.1109/IGARSS.2015.7326158 [21] Wang, M., Deng, W. H., 2018. Deep Visual Domain Adaptation: A Survey. Neurocomputing, 312: 135-153. https://doi.org/10.1016/j.neucom.2018.05.083 [22] Wang, Z. M., Du, B., Shi, Q., et al., 2019. Domain Adaptation with Discriminative Distribution and Manifold Embedding for Hyperspectral Image Classification. IEEE Geoscience and Remote Sensing Letters, 16(7): 1155-1159. https://doi.org/10.1109/LGRS.2018.2889967 [23] Xing, C., Ma, L., Yang, X. Q., 2016. Stacked Denoise Autoencoder Based Feature Extraction and Classification for Hyperspectral Images. Journal of Sensors, 2016: 1-10. https://doi.org/10.1155/2016/3632943 [24] Yang, H. L., Crawford, M. M., 2015. Spectral and Spatial Proximity-Based Manifold Alignment for MultitemporalHyperspectral Image Classification. IEEE Transactions on Geoscience and Remote Sensing, 54(1): 51-64. https://doi.org/10.1109/TGRS.2015.2449736 [25] Yeh, Y. R., Huang, C. H., Wang, Y. C., 2014. Heterogeneous Domain Adaptation and Classification by Exploiting the Correlation Subspace. IEEE Transactions on Image Processing, 23(5): 2009-2018. https://doi.org/10.1109/tip.2014.2310992 [26] Zhang, B.Y., Liu, Y.Y., Yuan, H.W., et al., 2018. A Joint Unsupervised Cross-Domain Model via Scalable Discriminative Extreme Learning Machine. Cognitive Computation, 10(4): 577-590. https://doi.org/10.1007/s12559-018-9555-z [27] Zhang, F., Du, B., Zhang, L.P., 2016. Scene Classification via a Gradient Boosting Random Convolutional Network Framework. IEEE Transactions on Geoscience and Remote Sensing, 54(3): 1793-1802. https://doi.org/10.1109/TGRS.2015.2488681 [28] Zhang, L., 2019. Transfer Adaptation Learning: A Decade Survey. In: Computing Research Repository. arXiv preprint. arXiv: abs/1903.04687 [29] Zhang, X., Yu, F. X., Chang, S. F., et al., 2015. Deep Transfer Network: Unsupervised Domain Adaptation: arXiv: 1503.00591[cs. CV]. https://arxiv.org/abs/1503.00591 [30] Zheng, G.Z., Le, X.D., Wang, H.P., et al., 2017. Inversion of Water Depth from WorldView-02 Satellite Imagery Based on BP and RBF Neural Network. Earth Science, 42(12): 2345-2353(in Chinese with English abstract). http://www.researchgate.net/publication/322615468_Inversion_of_Water_Depth_from_WorldView-02_Satellite_Imagery_Based_on_BP_and_RBF_Neural_Network [31] Zhuang, F., Cheng, X., Luo, P., 2015. Supervised Representation Learning: Transfer Learning with Deep Autoencoders. In: Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, IJCAI 2015. AAAI Press, Palo Alto CA, 4119-4125. [32] 程国轩, 牛瑞卿, 张凯翔, 等, 2018. 基于卷积神经网络的高分遥感影像露天采矿场识别. 地球科学, 43(增刊2): 256-262. doi: 10.3799/dqkx.2018.987 [33] 高常鑫, 桑农, 2014. 基于深度学习的高分辨率遥感影像目标检测. 测绘通报, (S1): 108-111. https://www.cnki.com.cn/Article/CJFDTOTAL-CHTB2014S1025.htm [34] 李文凯, 张唯, 秦家豪, 等, 2020. 基于DEM和高分辨率遥感影像的"膨胀-融合"式地表沟壑提取. 地球科学, 45(6): 1948-1955. doi: 10.3799/dqkx.2020.004 [35] 刘大伟, 韩玲, 韩晓勇, 2016. 基于深度学习的高分辨率遥感影像分类研究. 光学学报, 36(4): 0428001. https://www.cnki.com.cn/Article/CJFDTOTAL-GXXB201604039.htm [36] 郑贵洲, 乐校冬, 王红平, 等, 2017. 基于WorldView-02高分影像的BP和RBF神经网络遥感水深反演. 地球科学, 42(12): 2345-2353. doi: 10.3799/dqkx.2017.552 -
点击查看大图
图(5) / 表(3)
计量
- 文章访问数: 462
- HTML全文浏览量: 226
- PDF下载量: 33
- 被引次数: 0
