Algorithm2Domain - Integrative Domain Adaptation Benchmarking

“I am a data scientist focused on model development. How well does my model generalize to a different data set? What if that data set is from a completely different statistical distribution? Or even from a completely different domain? Can I test my model on many different domain-specific datasets?” “I work with domain data. How do I find the best model among so many out there to generalize well to my case? Can I test many models really fast on my data? Or on some open data which seems pretty close to the one I am still working to collect?” Algorithm2Domain is a meta-repository to facilitate finding the answers. Our goal is to aggregate existing algorithms and benchmarking suits, and develop integrative pipelines for mix-and-match cross-domain benchmarking. The data sources suitable for the benchmarking will be aggregated as pointers. The main focus is put on open data sets, but in some cases (e.g. for some medical data sets) additional formalities may be required to access the data. This repository will contain a integrative benchmarking suite, that connects the datasets, models, domain adaptation algorithms, few-shot approaches and datasets from various domain adaptation benchmarking suites. For now our seed benchmarking suite is ADATime by by: Mohamed Ragab*, Emadeldeen Eldele*, Wee Ling Tan, Chuan-Sheng Foo, Zhenghua Chen☨, Min Wu, Chee Kwoh, Xiaoli Li. In the Wiki we aggregate knowledge regarding domain adaptation and related topics: generalizability, out-of-distribution performance etc., as well as different methods to improve the algorithm design and to efficiently adapt it to the new domain.