Skip to content

Best transfer learning and domain adaptation resources (papers, tutorials, datasets, etc.)

Notifications You must be signed in to change notification settings

domainadaptation/awesome-transfer-learning

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 

Repository files navigation

Awesome Transfer Learning

A list of awesome papers and cool resources on transfer learning, domain adaptation and domain-to-domain translation in general! As you will notice, this list is currently mostly focused on domain adaptation (DA) and domain-to-domain translation, but don't hesitate to suggest resources in other subfields of transfer learning. I accept pull requests.

Table of Contents

Tutorials and Blogs

Papers

Papers are ordered by theme and inside each theme by publication date (submission date for arXiv papers). If the network or algorithm is given a name in a paper, this one is written in bold before the paper's name.

Surveys

Deep Transfer Learning

Transfer of deep learning models.

Fine-tuning approach

Feature extraction (embedding) approach

Multi-task learning

Policy transfer for RL

Few-shot transfer learning

Meta transfer learning

Applications

Medical imaging:

Robotics

Unsupervised Domain Adaptation

Transfer between a source and a target domain. In unsupervised domain adaptation, only the source domain can have labels.

Theory

Adversarial methods

Learning a latent space

Image-to-Image translation

Multi-source adaptation

Temporal models (videos)

Optimal Transport

Embedding methods

Kernel methods

Autoencoder approach

Subspace Learning

Self-Ensembling methods

Other

Semi-supervised Domain Adaptation

All the source points are labelled, but only few target points are.

General methods

Subspace learning

Copulas methods

Few-shot Supervised Domain Adaptation

Only a few target examples are available, but they are labelled

Adversarial methods

Embedding methods

Applied Domain Adaptation

Domain adaptation applied to other fields

Physics

Audio Processing

Datasets

Image-to-image

  • MNIST vs MNIST-M vs SVHN vs Synth vs USPS: digit images
  • GTSRB vs Syn Signs : traffic sign recognition datasets, transfer between real and synthetic signs.
  • NYU Depth Dataset V2: labeled paired images taken with two different cameras (normal and depth)
  • CelebA: faces of celebrities, offering the possibility to perform gender or hair color translation for instance
  • Office-Caltech dataset: images of office objects from 10 common categories shared by the Office-31 and Caltech-256 datasets. There are in total four domains: Amazon, Webcam, DSLR and Caltech.
  • Cityscapes dataset: street scene photos (source) and their annoted version (target)
  • UnityEyes vs MPIIGaze: simulated vs real gaze images (eyes)
  • CycleGAN datasets: horse2zebra, apple2orange, cezanne2photo, monet2photo, ukiyoe2photo, vangogh2photo, summer2winter
  • pix2pix dataset: edges2handbags, edges2shoes, facade, maps
  • RaFD: facial images with 8 different emotions (anger, disgust, fear, happiness, sadness, surprise, contempt, and neutral). You can transfer a face from one emotion to another.
  • VisDA 2017 classification dataset: 12 categories of object images in 2 domains: 3D-models and real images.
  • Office-Home dataset: images of objects in 4 domains: art, clipart, product and real-world.

Text-to-text

Results

The results are indicated as the prediction accuracy (in %) in the target domain after adapting the source to the target. For the moment, they only correspond to the results given in the original papers, so the methodology may vary between each paper and these results must be taken with a grain of salt.

Digits transfer (unsupervised)

Source
Target
MNIST
MNIST-M
Synth
SVHN
MNIST
SVHN
SVHN
MNIST
MNIST
USPS
USPS
MNIST
SA 56.90 86.44 ? 59.32 ? ?
DANN 76.66 91.09 ? 73.85 ? ?
CoGAN ? ? ? ? 91.2 89.1
DRCN ? ? 40.05 81.97 91.80 73.67
DSN 83.2 91.2 ? 82.7 ? ?
DTN ? ? 90.66 79.72 ? ?
PixelDA 98.2 ? ? ? 95.9 ?
ADDA ? ? ? 76.0 89.4 90.1
UNIT ? ? ? 90.53 95.97 93.58
GenToAdapt ? ? ? 92.4 95.3 90.8
SBADA-GAN 99.4 ? 61.1 76.1 97.6 95.0
DAassoc 89.47 91.86 ? 97.60 ? ?
CyCADA ? ? ? 90.4 95.6 96.5
I2I ? ? ? 92.1 95.1 92.2
DIRT-T 98.7 ? 76.5 99.4 ? ?
DeepJDOT 92.4 ? ? 96.7 95.7 96.4

Challenges

Libraries

No good library for the moment (as far as I know). If you're interested in a project of creating a generic transfer learning/domain adaptation library, please let me know.

About

Best transfer learning and domain adaptation resources (papers, tutorials, datasets, etc.)

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published