Tutorial on Domain Adaptation and Transfer Learning
September 7, 2014
Introduction
-
A Survey on Transfer Learning. S. J. Pan, Q. Yang. IEEE Trans. on Knowledge and Data Engineering, 22(10): 1345-1359, 2010. http://www1.i2r.a-star.edu.sg/~jspan/publications/TLsurvey_0822.pdf
-
Transfer learning. L. Torrey and J. Shavlik. In Handbook of Research on Machine Learning Applications and Trends: Algorithms,Methods, and Techniques. IGI Global, 2009. http://pages.cs.wisc.edu/~ltorrey/papers/torrey_chapter09.pdf
-
Learning Categories from Few Examples with Multi Model Knowledge Transfer. T. Tommasi, F. Orabona, B. Caputo. IEEE Trans. Pattern Analysis and Machine Intelligence, 36(5):928-941, 2014. http://homes.esat.kuleuven.be/~ttommasi/papers/TTommasi_PAMI_06620871.pdf
Reweight/Instance Based Methods
-
Direct Importance Estimation with Model Selection and Its Application to Covariate Shift Adaptation. M. Sugiyama, S. Nakajima, H. Kashima, P. von Buenau, M. Kawanabe. NIPS 2007. http://papers.nips.cc/paper/3248-direct-importance-estimation-with-model-selection-and-its-application-to-covariate-shift-adaptation.pdf
code: http://sugiyama-www.cs.titech.ac.jp/~sugi/software/KLIEP/
-
Discriminative Learning for Differing Training and Test Distributions. S. Bickel, M. Brűckner, T. Scheffer. ICML 2007. http://machinelearning.wustl.edu/mlpapers/paper_files/icml2007_BickelBS07.pdf
-
Correcting Sample Selection Bias by Unlabeled Data. J. Huang, A. Gretton, K. M. Borgwardt, B. Schlkopf, A.J. Smola. NIPS, 2006. http://machinelearning.wustl.edu/mlpapers/paper_files/NIPS2006_915.pdf
-
Covariate Shift by Kernel Mean Matching. A. Gretton, A. Smola, J.Huang, M. Schmittfull, K. Borgwardt, B. Scholkopf.NIPS 2007 (and book chapter) http://www.cs.cmu.edu/~arthurg/talks.html
code: http://www.cs.cmu.edu/~arthurg/papers/covariateShiftChapter.pdf
Model Adaptation
-
Safety in Numbers: Learning Categories from Few Examples with Multi Model Knowledge Transfer. T. Tommasi, F. Orabona, B. Caputo, CVPR 2010. http://homes.esat.kuleuven.be/~ttommasi/papers/1963.pdf
code: http://homes.esat.kuleuven.be/~ttommasi/source_code_CVPR10.html
-
Multiclass Transfer Learning from Unconstrained Priors, L. Jie, T. Tommasi, B. Caputo, ICCV 2011. http://homes.esat.kuleuven.be/~ttommasi/papers/1088.pdf
code: http://homes.esat.kuleuven.be/~ttommasi/source_code_ICCV11.html
-
Domain Adaptation from Multiple Sources via Auxiliary Classifiers. L. Duan, I. W. Tsang, D. Xu, T. Chua. ICML 2009
-
Domain Adaptation from Multiple Sources: A Domain-Dependent Regularization Approach. L. Duan, D. Xu, I. W. Tsang. T-NNLS, 2012
-
Exploiting Web Images for Event Recognition in Consumer Videos: A Multiple Source Domain Adaptation Approach. L. Duan, D. Xu, S. Chang. CVPR 2012 http://www.lxduan.info/papers/DuanCVPR2012.pdf
Model & Feature Adaptation
-
Domain Transfer SVM for video concept detection. L. Duan, D. Xu, I. W. Tsang. CVPR 2009. http://www.lxduan.info/papers/DuanCVPR2009.pdf
-
Domain Transfer Multiple Kernel Learning. Lixin Duan, I. W. Tsang, D. Xu. PAMI 2012. http://www.lxduan.info/papers/DuanTPAMI2012a.pdf
-
Visual Event Recognition in Videos by Learning from Web Data. L. Duan, I W.. Tsang, D. Xu, J. Luo. CVPR 2010, PAMI 2012
http://www.lxduan.info/papers/DuanCVPR2010.pdf
http://www.lxduan.info/papers/DuanTPAMI2012b.pdf
-
Efficient Learning of Domain-invariant Image Representations. J. Hoffman, E. Rodner, J. Donahue, K. Saenko, T. Darrell. ICLR 2013. http://www.eecs.berkeley.edu/~jhoffman/papers/Hoffman_ICLR2013.pdf
Feature & Projection Based Approaches
-
Domain Adaptation by Transfer Component Analysis. S. J. Pan, I. W. Tsang, J. T. Kwok, Q. Yang. TNN, 2011.
http://datam.i2r.a-star.edu.sg/papers/Sino/TNN11_DomainAdaptationviaTransferComponentAnalysis.pdf
-
Transfer Feature Learning with Joint Distribution Adaptation. M. Long, J. Wang, G. Ding, J. Sun, P. S. Yu, ICCV 2013. http://learn.tsinghua.edu.cn:8080/2011310560/publications/joint-iccv14.pdf
-
Domain Adaptation with Structural Correspondence Learning. J. Blitzer, R. McDonald, F. Pereira. EMNLP 2006. http://john.blitzer.com/papers/emnlp06.pdf
-
Biographies, Bollywood, Boom-boxes, and Blenders: Domain Adaptation for Sentiment Classification. J. Blitzer, M. Dredze, F. Pereira. ACL, 2007. http://john.blitzer.com/papers/sentiment_domain.pdf
-
Frustratingly Easy Domain Adaptation. H. Daumé III, ACL 2007. http://hal3.name/docs/daume07easyadapt.pdf
-
A Co-regularization Based Semi-supervised Domain Adaptation. A. Kumar, A. Saha, H. Daumé III, NIPS 2010. http://hal3.name/docs/daume10coreg.pdf
-
Domain Adaptation for Object Recognition: An Unsupervised Approach. R. Gopalan, R. Li, and R. Chellappa. ICCV, 2011.
http://www.umiacs.umd.edu/~raghuram/Publications/2011_ICCV_DomainAdaptation.pdf
-
Geodesic Flow Kernel for Unsupervised Domain Adaptation. B. Gong, Y. Shi, F. Sha, and K. GraumanCVPR 2012 http://www-scf.usc.edu/~boqinggo/Paper/GFK_cvpr12.pdf
-
Connecting the Dots with Landmarks: Discriminatively Learning Domain-Invariant Features for Unsupervised Domain Adaptation. B. Gong, K. Grauman, and F. Sha. ICML 2013 http://www-scf.usc.edu/~boqinggo/Paper/landmark.pdf
-
Unsupervised Visual Domain Adaptation Using Subspace Alignment. Basura Fernando, Amaury Habrard, Marc Sebban and Tinne Tuytelaars. ICCV 2013 http://homes.esat.kuleuven.be/~bfernand/papers/SA_ICCV_2013.pdf
-
Adapting Visual Category Models to New Domains. K. Saenko, B. Kulis, M. Fritz and T. Darrell. ECCV 2010. http://www.cs.uml.edu/~saenko/saenko_eccv_2010.pdf
Dictionary Learning & Avoid Dictionary
-
Subspace Interpolation via Dictionary Learning for Unsupervised Domain Adaptation. J. Ni, Q. Qiu, R. Chellappa.CVPR 2013 https://www.cs.umd.edu/~qiu/pub/subspace-cvpr13.pdf
-
Generalized Domain-Adaptive Dictionaries. S. Shekhar, V. M. Patel, H. V. Nguyen, R. Chellappa. CVPR 2013. http://www.umiacs.umd.edu/~hien/DomainAdaptDict.pdf
-
Frustratingly Easy NBNN Domain Adaptation. T. Tommasi, B. Caputo, ICCV 2013. http://homes.esat.kuleuven.be/~ttommasi/papers/TTommasi_ICCV_2013.pdf
Deep Learning
-
DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition. J. Donahue, Y. Jia, O. Vinyals, J. Hoffman, N. Zhang, E. Tzeng, T. Darrell. ICML 2014. http://arxiv.org/pdf/1310.1531.pdf
-
Learning and Transferring Mid-Level Image Representations using Convolutional Neural Networks. M. Oquab, L. Bottou, I. Laptev, J. Sivic. CVPR 2014 http://www.di.ens.fr/~josef/publications/oquab14.pdf
Iterative Adaptation
-
Domain Adaptation Problems: A DASVM Classification Technique and a Circular Validation Strategy. L. Bruzzone and M. Marconcini. PAMI, 32(5):770-787, 2010. http://rslab.disi.unitn.it/papers/R82-PAMI.pdf
-
Iterative Self-Labeling Domain Adaptation for Linear Structured Image Classification. A. Habrard, J.P. Peyrache, M. Sebban. International Journal on Artificial Intelligence Tools. 22(5), 2013. http://perso.univ-st-etienne.fr/habrarda/ARTICLES/IJAIT13.pdf
Detection
-
Transfer Learning by Borrowing Examples. J. Lim , R. Salakhutdinov, A. Torralba. NIPS 2012. http://www.cs.toronto.edu/~rsalakhu/papers/lst_nips.pdf
-
Enhancing Exemplar SVMs using Part Level Transfer Regularization. Y. Aytar, A. Zisserman, BMVC 2012. http://www.robots.ox.ac.uk/~vgg/publications/2012/Aytar12/aytar12.pdf
-
Tabula Rasa: Model Transfer for Object Category Detection. Y. Aytar, A. Zisserman, ICCV 2011. http://www.robots.ox.ac.uk/~vgg/publications/2011/Aytar11/aytar11.pdf
code: http://www.robots.ox.ac.uk/~vgg/software/tabularasa/
-
Large-scale knowledge transfer for object localization in ImageNet. M. Guillaumin, V. Ferrari. CVPR 2012. http://groups.inf.ed.ac.uk/calvin/Publications/guillaumin12cvpr.pdf
Boosting
-
Boosting for Unsupervised Domain Adaptation. A. Habrard, J.P. Peyrache and M. Sebban. ECML-PKDD 2013 http://www.ecmlpkdd2013.org/wp-content/uploads/2013/07/197.pdf
-
Non-Linear Domain Adaptation with Boosting, C. Becker, C. M. Christoudias, P. Fua. NIPS 2013. http://infoscience.epfl.ch/record/188474/files/Becker2013NIPS.pdf
-
Boosting for transfer learning with multiple sources, Y. Yao, G. Doretto, CVPR10. http://www.csee.wvu.edu/~gidoretto/publications/yaoD10cvpr.pdf
Theory
-
A theory of learning from different domains. Shai ben-David et al. http://web.eecs.umich.edu/~kulesza/pubs/adapt_mlj10.pdf
-
Domain Adaptation–Can Quantity Compensate for Quality? Shai Ben-David et al. https://cs.uwaterloo.ca/~shai/ISAIM12.pdf
-
Impossibility theorems for Domain Adaptation. Shai ben-David et al. http://www.cs.toronto.edu/~tl/papers/danfl.pdf
-
Domain Adaptation under Target and Conditional Shift. Kun Zhang, B. Sholkopf et al. http://people.tuebingen.mpg.de/kzhang/Target_shift_ICML13.pdf
-
Stability and Hypothesis Transfer Learning. I. Kuzborskij, F. Orabona. http://idiap.ch/~ikuzbor/pub/icml13.pdf
-
Domain Adaptation as Learning with Auxiliary information. R. Urner S. Ben David workshop NIPS 2013 https://docs.google.com/file/d/0B6_yFtMDxz8HZXZoQllpaEdzZWs/edit
Some useful references.