Accepted papers
*All times are displayed in PDT
Monday, May 3
01:00 – 03:00 AM | Poster
What Makes Instance Discrimination Good for Transfer Learning?
Nanxuan Zhao, Zhirong Wu, Rynson W. H. Lau, Stephen Lin
09:00 – 11:00 AM | Poster
Learning-based Support Estimation in Sublinear Time
Talya Eden, Piotr Indyk, Shyam Narayanan, Ronitt Rubinfeld, Sandeep Silwal, Tal Wagner
Spotlight Session: May 4
Parameter Efficient Multimodal Transformers for Video Representation Learning
Sangho Lee, Youngjae Yu, Gunhee Kim, Thomas Breuel, Jan Kautz, Yale Song
Shapley Explainability on the Data Manifold
Christopher Frye, Damien de Mijolla, Tom Begley, Laurence Cowton, Megan Stanley, Ilya Feige
InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective
Boxin Wang, Shuohang Wang, Yu Cheng, Zhe Gan, Ruoxi Jia, Bo Li, Jingjing Liu
12:35 – 12:45 PM | Spotlight
Systematic Generalisation with Group Invariant Predictions
Faruk Ahmed, Yoshua Bengio, Harm van Seijen, Aaron Courville
Poster Session: May 4
05:00 – 07:00 PM | Poster
VA-RED$^2$: Video Adaptive Redundancy Reduction
Bowen Pan, Rameswar Panda, Camilo Luciano Fosco, Chung-Ching Lin, Alex J Andonian, Yue Meng, Kate Saenko, Aude Oliva, Rogerio Feris
SCoRe: Pre-Training for Context Representation in Conversational Semantic Parsing
Tao Yu, Rui Zhang, Alex Polozov, Christopher Meek, Ahmed Hassan Awadallah
DEBERTA: Decoding-Enhanced BERT with Disentangled Attention
Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen
Learning a Latent Simplex in Input Sparsity Time
Ainesh Bakshi, Chiranjib Bhattacharyya, Ravi Kannan, David Woodruff, Samson Zhou
Spotlight Session: May 6
Taking Notes on the Fly Helps Language Pre-Training
Qiyu Wu, Chen Xing, Yatao Li, Guolin Ke, Di He, Tie-Yan Liu
MixKD: Towards Efficient Distillation of Large-scale Language Models
Kevin J Liang, Weituo Hao, Dinghan Shen, Yufan Zhou, Weizhu Chen, Changyou Chen, Lawrence Carin
Rethinking Positional Encoding in Language Pre-training
Tuesday, May 4
01:00 – 03:00 AM | Poster
GraphCodeBERT: Pre-training Code Representations with Data Flow
Daya Guo, Shuo Ren, Shuai Lu, Zhangyin Feng, Duyu Tang, Shujie Liu, Long Zhou, Nan Duan, Alexey Svyatkovskiy, Shengyu Fu, Michele Tufano, Shao Kun Deng, Colin Clement, Dawn Drain, Neel Sundaresan, Jian Yin, Daxin Jiang, Ming Zhou
09:00 – 11:00 AM | Poster
Self-Supervised Learning of Compressed Video Representations
Youngjae Yu, Sangho Lee, Gunhee Kim, Yale Song
Data-Efficient Reinforcement Learning with Self-Predictive Representations
Max Schwarzer, Ankesh Anand, Rishab Goel, R Devon Hjelm, Aaron Courville, Philip Bachman
Spotlight Session: May 6
Provable Rich Observation Reinforcement Learning with Combinatorial Latent States
Dipendra Misra, Qinghua Liu, Chi Jin, John Langford
Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval
Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul N. Bennett, Junaid Ahmed, Arnold Overwikj
Systematic Generalisation with Group Invariant Predictions
Faruk Ahmed, Yoshua Bengio, Harm van Seijen, Aaron Courville
Spotlight Session: May 3
01:28 – 01:38 PM | Spotlight
Learning-based Support Estimation in Sublinear Time
Talya Eden, Piotr Indyk, Shyam Narayanan, Ronitt Rubinfeld, Sandeep Silwal, Tal Wagner
Poster Session: May 3
05:00 – 07:00 PM | Poster
CoDA: Contrast-enhanced and Diversity-promoting Data Augmentation for Natural Language Understanding
Yanru Qu, Dinghan Shen, Yelong Shen, Sandra Sajeev, Weizhu Chen, Jiawei Han
Knowledge Distillation as Semiparametric Inference
Tri Dao, Govinda M Kamath, Vasilis Syrgkanis, Lester Mackey
Debiasing Concept-based Explanations with Causal Analysis
Mohammad Taha Bahadori, David Heckerman
Aligning AI with Shared Human Values
Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Li, Dawn Song, Jacob Steinhardt
Wednesday, May 5
01:00 – 03:00 AM | Poster
Active Contrastive Learning of Audio-Visual Video Representations
Shuang Ma, Zhaoyang Zeng, Daniel McDuff, Yale Song
Dance Revolution: Long-Term Dance Generation with Music via Curriculum Learning
Ruozi Huang, Huang Hu, Wei Wu, Kei Sawada, Mi Zhang, Daxin Jiang
Return-Based Contrastive Representation Learning for Reinforcement Learning
Guoqing Liu, Chuheng Zhang, Li Zhao, Tao Qin, Jinhua Zhu, Li Jian, Nenghai Yu, Tie-Yan Liu
Byzantine-Resilient Non-Convex Stochastic Gradient Descent
Zeyuan AllenZhu, Faeze Ebrahimianghazani, Jerry Li, Dan Alistarh
09:00 – 11:00 AM | Poster
Learning to Represent Action Values as a Hypergraph on the Action Vertices
Arash Tavakoli, Mehdi Fatemi, Petar Kormushev
Optimism in Reinforcement Learning with Generalized Linear Function Approximation
Yining Wang, Ruosong Wang, Simon Shaolei Du, Akshay Krishnamurthy
SEED: Self-supervised Distillation For Visual Representation
Zhiyuan Fang, Jianfeng Wang, Lijuan Wang, Lei Zhang, Yezhou Yang, Zicheng Liu
05:00 – 07:00 PM | Poster
Filtered Inner Product Projection for Crosslingual Embedding Alignment
Vin Sachidananda, Ziyi Yang, Chenguang Zhu
Economic Hyperparameter Optimization With Blended Search Strategy
Chi Wang, Qingyun Wu, Silu Huang, Amin Saied
ALFWorld: Aligning Text and Embodied Environments for Interactive Learning
Mohit Shridhar, Xingdi Yuan, Marc-Alexandre Cote, Yonatan Bisk, Adam Trischler, Matthew Hausknecht
AdaFuse: Adaptive Temporal Fusion Network for Efficient Action Recognition
Yue Meng, Rameswar Panda, Chung-Ching Lin, Prasanna Sattigeri, Leonid Karlinsky, Kate Saenko, Aude Oliva, Rogerio Feris
AdaSpeech: Adaptive Text to Speech for Custom Voice
Mingjian Chen, Xu Tan, Bohan Li, Yanqing Liu, Tao Qin, Sheng Zhao, Tie-Yan Liu
Revisiting Dynamic Convolution via Matrix Decomposition
Yunsheng Li, Yinpeng Chen, Xiyang Dai, Mengchen Liu, Dongdong Chen, Ye Yu, Lu Yuan, Zicheng Liu, Mei Chen, Nuno Vasconcelos
07:25 – 07:35 PM | Spotlight
Large Scale Image Completion via Co-Modulated Generative Adversarial Networks
Shengyu Zhao, Jonathan Cui, Yilun Sheng, Yue Dong, Xiao Liang, Eric I-Chao Chang, Yan Xu
Poster Session: May 6
07:55 – 08:10 PM | Oral
Deformable DETR: Deformable Transformers for End-to-End Object Detection
Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, Jifeng Dai
Poster Session: May 6
Thursday, May 6
01:00 – 03:00 AM | Poster
Do not Let Privacy Overbill Utility: Gradient Embedding Perturbation for Private Learning
Da Yu, Huishuai Zhang, Wei Chen, Tie-Yan Liu
IOT: Instance-wise Layer Reordering for Transformer Structures
Jinhua Zhu, Lijun Wu, Yingce Xia, Shufang Xie, Tao Qin, Wengang Zhou, Houqiang Li, Tie-Yan Liu
Deformable DETR: Deformable Transformers for End-to-End Object Detection
Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, Jifeng Dai
Oral Session: May 5
09:00 – 11:00 AM | Poster
Initialization and Regularization of Factorized Neural Layers
Mikhail Khodak, Neil A. Tenenholtz, Lester Mackey, Nicolo Fusi
Learning to Recombine and Resample Data For Compositional Generalization
Ekin Akyürek, Afra Feyza Akyürek, Jacob Andreas
Adversarial score matching and improved sampling for image generation
Alexia Jolicoeur-Martineau, Rémi Piché-Taillefer, Ioannis Mitliagkas, Remi Tachet des Combes
Improving Zero-Shot Voice Style Transfer via Disentangled Representation Learning
Siyang Yuan, Pengyu Cheng, Ruiyi Zhang, Weituo Hao, Zhe Gan, Lawrence Carin
12:40 – 12:50 PM | Spotlight
Data-Efficient Reinforcement Learning with Self-Predictive Representations
Max Schwarzer, Ankesh Anand, Rishab Goel, R Devon Hjelm, Aaron Courville, Philip Bachman
Poster Session: May 4
05:00 – 07:00 PM | Poster
Representing Partial Programs with Blended Abstract Semantics
Maxwell Nye, Yewen Pu, Matthew Bowers, Jacob Andreas, Joshua B. Tenenbaum, Armando Solar-Lezama
FastSpeech 2: Fast and High-Quality End-to-End Text to Speech
Yi Ren, Chenxu Hu, Xu Tan, Tao Qin, Sheng Zhao, Zhou Zhao, Tie-Yan Liu
Large Scale Image Completion via Co-Modulated Generative Adversarial Networks
Shengyu Zhao, Jonathan Cui, Yilun Sheng, Yue Dong, Xiao Liang, Eric I-Chao Chang, Yan Xu
Spotlight Session: May 5
DynaTune: Dynamic Tensor Program Optimization in Deep Neural Network Compilation
Minjia Zhang, Menghao Li, Chi Wang, Mingqin Li
08:58 – 09:08 PM | Spotlight
Learning a Latent Simplex in Input Sparsity Time
Ainesh Bakshi, Chiranjib Bhattacharyya, Ravi Kannan, David Woodruff, Samson Zhou
Poster Session: May 3