Skip to main content
Log in

Learning discriminative feature via a generic auxiliary distribution for unsupervised domain adaptation

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Traditional methods for unsupervised domain adaptation often leverage a projection matrix or a neural network as the feature extractor or classifier, where the feature extractor shared by the source and target domains enables the sample distributions to be aligned in the feature space, and simultaneously makes the source domain features separability enough for the classifier. However, only the alignment of both domains is not enough because the inter-class distance of some categories in the target domain may be too small, i.e., the feature separability is poor, which often leads to the bad condition that some samples are projected to the classification boundaries and thus misclassified. To solve this problem, we propose a pluggable generic auxiliary distribution (GAD) module for target domain in this paper. The proposed GAD module can iteratively refine the prediction of the target domain samples to increase the separability of the learned features, thereby increasing the distance between features of different categories. This operation can finally reduce the possibility of the target domain samples falling near the classification boundary, and leads to the improvement of classification accuracy for the target domain. Extensive experiments on several popular datasets are conducted, and the results demonstrate the effectiveness of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Ben-David S, Blitzer J, Crammer K, Kulesza A, Pereira F, Vaughan JW (2010) A theory of learning from different domains. Mach Learn 79:151–175

    Article  MathSciNet  Google Scholar 

  2. Bousmalis K, Silberman N, Dohan D, Erhan D, Krishnan D (2017) Unsupervised pixel-level domain adaptation with generative adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Honolulu, USA, pp 3722–3731

  3. Bousmalis K, Trigeorgis G, Silberman N, Krishnan D, Erhan D (2016) Domain separation networks. In: Advances in neural information processing systems, Barcelona, Spain, pp 1–9

  4. Chen C, Xie W, Huang W, Rong Y, Ding X, Huang Y, Xu T, Huang J (2019) Progressive feature alignment for unsupervised domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Long Beach, CA, USA, pp 627–636

  5. Chen M, Xue H, Cai D (2019) Domain adaptation for semantic segmentation with maximum squares loss. In: International conference on computer vision, Seoul, Korea, pp 2090–2099

  6. Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L (2009) Imagenet: a large-scale hierarchical image database. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Miami, FL, USA, pp 248–255

  7. Fang J, Xu X, Liu H, Sun F (2019) Local receptive field based extreme learning machine with three channels for histopathological image classification. Int J Mach Learn Cybern 10:1437–1447

    Article  Google Scholar 

  8. Ganin Y, Lempitsky V (2015) Unsupervised domain adaptation by backpropagation. In: International conference on machine learning, Lille, France, pp 1180–1189

  9. Ghifary M, Kleijn WB, Zhang M, Balduzzi D, Li W (2016) Deep reconstruction-classification networks for unsupervised domain adaptation. In: European conference on computer vision, Amsterdam, The Netherlands, pp 597–613

  10. Haeusser P, Frerix T, Mordvintsev A, Cremers D (2017) Associative domain adaptation. In: International conference on computer vision, Venice, Italy, pp 2765–2773

  11. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Las Vegas, NV, USA, pp 770–778

  12. Hull J (1994) A database for handwritten text recognition research. IEEE Trans Pattern Anal Mach Intell 16(5):550–554

    Article  Google Scholar 

  13. Jia X, Jin Y, Su X, Hu Y (2019) Domain-invariant representation learning using an unsupervised domain adversarial adaptation deep neural network. Neurocomputing 355:209–220

    Article  Google Scholar 

  14. Khorshidpour Z, Tahmoresnezhad J, Hashemi S, Hamzeh A (2018) Domain invariant feature extraction against evasion attack. Int J Mach Learn Cybern 9:2093–2104

    Article  Google Scholar 

  15. Kim M, Sahu P, Gholami B, Pavlovic V (2019) Unsupervised visual domain adaptation: a deep max-margin Gaussian process approach. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Long Beach, CA, USA, pp 4380–4390

  16. Kim T, Jeong M, Kim S, Choi S, Kim C (2019) Diversify and match: a domain adaptive representation learning paradigm for object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Long Beach, CA, USA, pp 12456–12465

  17. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  18. Lin TY, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollár P, Zitnick CL (2014) Microsoft coco: common objects in context. In: European conference on computer vision, Zurich, Switzerland, pp 740–755

  19. Liu MY, Tuzel O (2016) Coupled generative adversarial networks. In: Advances in neural information processing systems, Barcelona, Spain, pp 1–9

  20. Long M, Cao Y, Wang J, Jordan M (2015) Learning transferable features with deep adaptation networks. In: International conference on machine learning, Lille, France, pp 97–105

  21. Lv F, Han M (2019) Hyperspectral image classification based on multiple reduced kernel extreme learning machine. Int J Mach Learn Cybern 10:3397–3405

    Article  Google Scholar 

  22. Van der Maaten L, Hinton G (2008) Visualizing data using t-SNE. J Mach Learn Res 9(11):2579–2605

    MATH  Google Scholar 

  23. Moiseev B, Konev A, Chigorin A, Konushin A (2013) Evaluation of traffic sign recognition methods trained on synthetically generated data. In: International conference on advanced concepts for intelligent vision systems, Poznan, Poland, pp 576–583

  24. Netzer Y, Wang T, Coates A, Bissacco A, Wu B, Ng AY (2011) Reading digits in natural images with unsupervised feature learning. In: Advances in neural information processing systems workshop, Granada Spain, pp 1–9

  25. Pinheiro PO (2018) Unsupervised domain adaptation with similarity learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Salt Lake City, Utah, USA, pp 8004–8013

  26. Saito K, Ushiku Y, Harada T (2017) Asymmetric tri-training for unsupervised domain adaptation. In: International conference on machine learning, Sydney, Australia, pp 2988–2997

  27. Saito K, Watanabe K, Ushiku Y, Harada T (2018) Maximum classifier discrepancy for unsupervised domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Salt Lake City, Utah, USA, pp 3723–3732

  28. Sankaranarayanan S, Balaji Y, Castillo CD, Chellappa R (2018) Generate to adapt: aligning domains using generative adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Salt Lake City, Utah, USA, pp 8503–8512

  29. Springenberg JT (2016) Unsupervised and semi-supervised learning with categorical generative adversarial networks. In: International conference on learning representations, San Juan, Puerto Rico, pp 1–20

  30. Stallkamp J, Schlipsing M, Salmen J, Igel C (2011) The German traffic sign recognition benchmark: a multi-class classification competition. In: International joint conference on neural networks, San Jose, CA, USA, pp 1453–1460

  31. Tzeng E, Hoffman J, Saenko K, Darrell T (2017) Adversarial discriminative domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Honolulu, HI, USA, pp 7167–7176

  32. Vu TH, Jain H, Bucher M, Cord M, Pérez P (2019) Advent: adversarial entropy minimization for domain adaptation in semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Long Beach, CA, USA, pp 2517–2526

  33. Wang Z, Xiao P, Tu W, Du B, Cheng Y (2019) Bi-adapting kernel learning for unsupervised domain adaptation. Neurocomputing 398:547–554

    Article  Google Scholar 

  34. Xie J, Girshick R, Farhadi A (2016) Unsupervised deep embedding for clustering analysis. In: International conference on machine learning, New York City, NY, USA, pp 478–487

  35. Yu C, Wang J, Chen Y, Qin X (2019) Transfer channel pruning for compressing deep domain adaptation models. Int J Mach Learn Cybern 10:3129–3144

    Article  Google Scholar 

  36. Zhang X, Zhang H, Lu J, Shao L, Yang J (2021) Target-targeted domain adaptation for unsupervised semantic segmentation. In: International conference on robotics and automation, Xi'an, China, pp 1–7

  37. Zhu JY, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: International conference on computer vision, Venice, Italy, pp 2223–2232

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China under Grant no. 61872187 and no. 62072246, in part by the Central Public-Interest Scientific Institution Basal Research Fund under Grant no. CAFYBB2019QD003, and in part by the Natural Science Foundation of Jiangsu Province under Grant no. BK20201306.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Haofeng Zhang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, Q., Zhang, H., Ye, Q. et al. Learning discriminative feature via a generic auxiliary distribution for unsupervised domain adaptation. Int. J. Mach. Learn. & Cyber. 13, 175–185 (2022). https://doi.org/10.1007/s13042-021-01381-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-021-01381-x

Keywords

Navigation