Abstract
Traditional methods for unsupervised domain adaptation often leverage a projection matrix or a neural network as the feature extractor or classifier, where the feature extractor shared by the source and target domains enables the sample distributions to be aligned in the feature space, and simultaneously makes the source domain features separability enough for the classifier. However, only the alignment of both domains is not enough because the inter-class distance of some categories in the target domain may be too small, i.e., the feature separability is poor, which often leads to the bad condition that some samples are projected to the classification boundaries and thus misclassified. To solve this problem, we propose a pluggable generic auxiliary distribution (GAD) module for target domain in this paper. The proposed GAD module can iteratively refine the prediction of the target domain samples to increase the separability of the learned features, thereby increasing the distance between features of different categories. This operation can finally reduce the possibility of the target domain samples falling near the classification boundary, and leads to the improvement of classification accuracy for the target domain. Extensive experiments on several popular datasets are conducted, and the results demonstrate the effectiveness of the proposed method.
Similar content being viewed by others
References
Ben-David S, Blitzer J, Crammer K, Kulesza A, Pereira F, Vaughan JW (2010) A theory of learning from different domains. Mach Learn 79:151–175
Bousmalis K, Silberman N, Dohan D, Erhan D, Krishnan D (2017) Unsupervised pixel-level domain adaptation with generative adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Honolulu, USA, pp 3722–3731
Bousmalis K, Trigeorgis G, Silberman N, Krishnan D, Erhan D (2016) Domain separation networks. In: Advances in neural information processing systems, Barcelona, Spain, pp 1–9
Chen C, Xie W, Huang W, Rong Y, Ding X, Huang Y, Xu T, Huang J (2019) Progressive feature alignment for unsupervised domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Long Beach, CA, USA, pp 627–636
Chen M, Xue H, Cai D (2019) Domain adaptation for semantic segmentation with maximum squares loss. In: International conference on computer vision, Seoul, Korea, pp 2090–2099
Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L (2009) Imagenet: a large-scale hierarchical image database. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Miami, FL, USA, pp 248–255
Fang J, Xu X, Liu H, Sun F (2019) Local receptive field based extreme learning machine with three channels for histopathological image classification. Int J Mach Learn Cybern 10:1437–1447
Ganin Y, Lempitsky V (2015) Unsupervised domain adaptation by backpropagation. In: International conference on machine learning, Lille, France, pp 1180–1189
Ghifary M, Kleijn WB, Zhang M, Balduzzi D, Li W (2016) Deep reconstruction-classification networks for unsupervised domain adaptation. In: European conference on computer vision, Amsterdam, The Netherlands, pp 597–613
Haeusser P, Frerix T, Mordvintsev A, Cremers D (2017) Associative domain adaptation. In: International conference on computer vision, Venice, Italy, pp 2765–2773
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Las Vegas, NV, USA, pp 770–778
Hull J (1994) A database for handwritten text recognition research. IEEE Trans Pattern Anal Mach Intell 16(5):550–554
Jia X, Jin Y, Su X, Hu Y (2019) Domain-invariant representation learning using an unsupervised domain adversarial adaptation deep neural network. Neurocomputing 355:209–220
Khorshidpour Z, Tahmoresnezhad J, Hashemi S, Hamzeh A (2018) Domain invariant feature extraction against evasion attack. Int J Mach Learn Cybern 9:2093–2104
Kim M, Sahu P, Gholami B, Pavlovic V (2019) Unsupervised visual domain adaptation: a deep max-margin Gaussian process approach. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Long Beach, CA, USA, pp 4380–4390
Kim T, Jeong M, Kim S, Choi S, Kim C (2019) Diversify and match: a domain adaptive representation learning paradigm for object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Long Beach, CA, USA, pp 12456–12465
LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324
Lin TY, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollár P, Zitnick CL (2014) Microsoft coco: common objects in context. In: European conference on computer vision, Zurich, Switzerland, pp 740–755
Liu MY, Tuzel O (2016) Coupled generative adversarial networks. In: Advances in neural information processing systems, Barcelona, Spain, pp 1–9
Long M, Cao Y, Wang J, Jordan M (2015) Learning transferable features with deep adaptation networks. In: International conference on machine learning, Lille, France, pp 97–105
Lv F, Han M (2019) Hyperspectral image classification based on multiple reduced kernel extreme learning machine. Int J Mach Learn Cybern 10:3397–3405
Van der Maaten L, Hinton G (2008) Visualizing data using t-SNE. J Mach Learn Res 9(11):2579–2605
Moiseev B, Konev A, Chigorin A, Konushin A (2013) Evaluation of traffic sign recognition methods trained on synthetically generated data. In: International conference on advanced concepts for intelligent vision systems, Poznan, Poland, pp 576–583
Netzer Y, Wang T, Coates A, Bissacco A, Wu B, Ng AY (2011) Reading digits in natural images with unsupervised feature learning. In: Advances in neural information processing systems workshop, Granada Spain, pp 1–9
Pinheiro PO (2018) Unsupervised domain adaptation with similarity learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Salt Lake City, Utah, USA, pp 8004–8013
Saito K, Ushiku Y, Harada T (2017) Asymmetric tri-training for unsupervised domain adaptation. In: International conference on machine learning, Sydney, Australia, pp 2988–2997
Saito K, Watanabe K, Ushiku Y, Harada T (2018) Maximum classifier discrepancy for unsupervised domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Salt Lake City, Utah, USA, pp 3723–3732
Sankaranarayanan S, Balaji Y, Castillo CD, Chellappa R (2018) Generate to adapt: aligning domains using generative adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Salt Lake City, Utah, USA, pp 8503–8512
Springenberg JT (2016) Unsupervised and semi-supervised learning with categorical generative adversarial networks. In: International conference on learning representations, San Juan, Puerto Rico, pp 1–20
Stallkamp J, Schlipsing M, Salmen J, Igel C (2011) The German traffic sign recognition benchmark: a multi-class classification competition. In: International joint conference on neural networks, San Jose, CA, USA, pp 1453–1460
Tzeng E, Hoffman J, Saenko K, Darrell T (2017) Adversarial discriminative domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Honolulu, HI, USA, pp 7167–7176
Vu TH, Jain H, Bucher M, Cord M, Pérez P (2019) Advent: adversarial entropy minimization for domain adaptation in semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Long Beach, CA, USA, pp 2517–2526
Wang Z, Xiao P, Tu W, Du B, Cheng Y (2019) Bi-adapting kernel learning for unsupervised domain adaptation. Neurocomputing 398:547–554
Xie J, Girshick R, Farhadi A (2016) Unsupervised deep embedding for clustering analysis. In: International conference on machine learning, New York City, NY, USA, pp 478–487
Yu C, Wang J, Chen Y, Qin X (2019) Transfer channel pruning for compressing deep domain adaptation models. Int J Mach Learn Cybern 10:3129–3144
Zhang X, Zhang H, Lu J, Shao L, Yang J (2021) Target-targeted domain adaptation for unsupervised semantic segmentation. In: International conference on robotics and automation, Xi'an, China, pp 1–7
Zhu JY, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: International conference on computer vision, Venice, Italy, pp 2223–2232
Acknowledgements
This work was supported by the National Natural Science Foundation of China under Grant no. 61872187 and no. 62072246, in part by the Central Public-Interest Scientific Institution Basal Research Fund under Grant no. CAFYBB2019QD003, and in part by the Natural Science Foundation of Jiangsu Province under Grant no. BK20201306.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Chen, Q., Zhang, H., Ye, Q. et al. Learning discriminative feature via a generic auxiliary distribution for unsupervised domain adaptation. Int. J. Mach. Learn. & Cyber. 13, 175–185 (2022). https://doi.org/10.1007/s13042-021-01381-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13042-021-01381-x