Microsoft at NeurIPS 2022

Differentially Private Model Compression

Fatemehsadat Mireshghallah, Arturs Backurs, Huseyin Inan, Lukas Wutschitz, Janardhan (Jana) Kulkarni

Self-explaining deep models with logic rule reasoning

Seungeon Lee, Xiting Wang, Sungwon Han, Xiaoyuan Yi, Xing Xie, Meeyoung Cha

Equivariant Networks for Zero-Shot Coordination

Darius Muglich, Christian Schroeder de Witt, Elise van der Pol, Shimon Whiteson, Jakob Nicolaus Foerster

Mask-based Latent Reconstruction for Reinforcement Learning

Tao Yu, Zhizheng Zhang, Cuiling Lan, Yan Lu, Zhibo Chen

Task-Agnostic Graph Explanations

Yaochen Xie, Sumeet Katariya, Xianfeng Tang, Edward Huang, Nikhil Rao, Karthik Subbian, Shuiwang Ji

On the Complexity of Adversarial Decision Making

Dylan J Foster, Alexander Rakhlin, Ayush Sekhari, Karthik Sridharan

Sample Optimality and All-for-all Strategies in Personalized Federated and Collaborative Learning

Mathieu Even, Laurent Massoulie, Kevin Scaman

Gradient Estimation with Discrete Stein Operators

Jiaxin Shi, Yuhao Zhou, Jessica Hwang, Michalis Titsias, Lester Mackey

Extreme Compression for Pre-trained Transformers Made Simple and Efficient

Xiaoxia Wu, Zhewei Yao, Minjia Zhang, Conglong Li, Yuxiong He

Museformer: Transformer with Fine- and Coarse-Grained Attention for Music Generation

Botao Yu, Peiling Lu, Rui Wang, Wei Hu, Xu Tan, Wei Ye, Shikun Zhang, Tao Qin, Tie-Yan Liu

Provably sample-efficient RL with side information about latent dynamics

Yao Liu, Dipendra Misra, Miro Dudík, Robert E. Schapire

Learning (Very) Simple Generative Models Is Hard

Sitan Chen, Jerry Li, Yuanzhi Li

Censored Quantile Regression Neural Networks for Distribution-Free Survival Analysis

Tim Pearce, Jong-Hyeon Jeong, Yichen Jia, Jun Zhu

Exponentially Improving the Complexity of Simulating the Weisfeiler-Lehman Test with Graph Neural Networks

Anders Aamand, Justin Y Chen, Piotr Indyk, Shyam Narayanan, Ronitt Rubinfeld, Nicholas Schiefer, Sandeep Silwal, Tal Wagner

Alignment-guided Temporal Attention for Video Action Recognition

Yizhou Zhao, Zhenyang Li, Xun Guo, Yan Lu

Maximum Class Separation as Inductive Bias in One Matrix

Tejaswi Kasarla, Gertjan J. Burghouts, Max van Spengler, Elise van der Pol, Rita Cucchiara, Pascal Mettes

VRL3: A Data-Driven Framework for Visual Deep Reinforcement Learning

Che Wang, Xufang Luo, Keith Ross, Dongsheng Li

Efficient and Effective Multi-task Grouping via Meta Learning on Task Combinations

Xiaozhuang Song, Shun Zheng, Wei Cao, James Yu, Jiang Bian

Incentivizing Combinatorial Bandit Exploration

Xinyan Hu, Dung Daniel Ngo, Aleksandrs Slivkins, Steven Wu

RKHS-SHAP: Shapley Values for Kernel Methods

Siu Lun Chau, Javier González, Dino Sejdinovic

Temporal Latent Bottleneck: Synthesis of Fast and Slow Processing Mechanisms in Sequence Learning

Aniket Didolkar, Kshitij Gupta, Anirudh Goyal, Nitesh B. Gundavarapu, Alex Lamb, Nan Rosemary Ke, Yoshua Bengio

LiteTransformerSearch: Training-free On-device Search for Efficient Autoregressive Language Models

Mojan Javaheripi, Shital Shah, Subhabrata (Subho) Mukherjee, Tomasz L. Religa, Caio C. T. Mendes, Gustavo H. de Rosa, Sébastien Bubeck, Farinaz Koushanfar, Debadeepta Dey

GLIPv2: Unifying Localization and Vision-Language Understanding

Haotian Zhang, Pengchuan Zhang, Xiaowei Hu, Yen-Chun Chen, Liunian Harold Li, Xiyang Dai, Lijuan Wang, Lu Yuan, Jenq-Neng Hwang, Jianfeng Gao

Robust Model Selection and Nearly-Proper Learning for GMMs

Jerry Li

Matching in Multi-arm Bandit with Collision

Yirui Zhang, Siwei Wang, Zhixuan Fang

Uni[MASK]: Unified Inference in Sequential Decision Problems

Micah Carroll, Orr Paradise, Jessy Lin, Raluca Georgescu, Mingfei Sun, Dave Bignell, Stephanie Milani, Katja Hofmann, Matthew Hausknecht, Anca Dragan, Sam Devlin

Missing Data Imputation and Acquisition with Deep Hierarchical Models and Hamiltonian Monte Carlo

Ignacio Peis, Chao Ma, Jose Miguel Hernandez Lobato

Fast and Scalable Homomorphically Encrypted Graph Convolutional Network Inference

Ran Ran, Wei Wang, Quan Gang, Jieming Yin, Nuo Xu, Wujie Wen

Contextual Squeeze-and-Excitation for Efficient Few-Shot Image Classification

Massimiliano Patacchiola, John F Bronskill, Aliaksandra Shysheya, Katja Hofmann, Sebastian Nowozin, Richard E Turner

Learning to Discover and Detect Objects

Vladimir Fomenko, Ismail Elezi, Deva Ramanan, Aljosa Osep, Laura Leal-Taixé

Unsupervised Learning of Group Invariant and Equivariant Representations

Robin Winter, Marco Bertolini, Tuan Le, Frank Noé, Djork-Arné Clevert

Visual Concepts Tokenization

Tao Yang, Yuwang Wang, Yan Lu, Nanning Zheng

Unpacking Reward Shaping: Understanding the Benefits of Reward Engineering on Sample Complexity

Abhishek Gupta, Aldo Pacchiano, Yuexiang Zhai, Sham Kakade, Sergey Levine

On the Statistical Efficiency of Reward-Free Exploration in Non-Linear RL

Jinglin Chen, Aditya Modi, Akshay Krishnamurthy, Nan Jiang, Alekh Agarwal

Debiased Machine Learning without Sample-Splitting for Stable Estimators

Qizhao Chen, Vasilis Syrgkanis, Morgane Austern

The Power and Limitation of Pretraining-Finetuning for Linear Regression under Covariate Shift

Jingfeng Wu, Difan Zou, Vladimir Braverman, Quanquan Gu, Sham Kakade

Fault-Aware Neural Code Rankers

Jeevana Priya Inala, Chenglong Wang, Mei Yang, Andres Codas, Mark Encarnación, Shuvendu Lahiri, Madan Musuvathi, Jianfeng Gao

Peripheral Vision Transformer

Juhong Min, Yucheng Zhao, Chong Luo, Minsu Cho

RTFormer: Efficient Design for Real-Time Semantic Segmentation with Transformer

Jian Wang, Chenhui Gou, Qiman Wu, Haocheng Feng, Junyu Han, Errui Ding, Jingdong Wang

M³ViT: Mixture-of-Experts Vision Transformer for Efficient Multi-task Learning with Model-Accelerator Co-design

Hanxue Liang, Zhiwen Fan, Rishov Sarkar, Ziyu Jiang, Tianlong Chen, Yu Cheng, Cong Hao, Zhangyang Wang

Risk Bounds of Multi-Pass SGD for Least Squares in the Interpolation Regime

Difan Zou, Jingfeng Wu, Vladimir Braverman, Quanquan Gu, Sham Kakade

Understanding the Eluder Dimension

Gene Li, Pritish Kamath, Dylan Foster, Nathan Srebro

Online Algorithms for the Santa Claus Problem

MohammadTaghi Hajiaghayi, MohammadReza Khani, Debmalya Panigrahi, Max Springer

Best of Both Worlds Model Selection

Aldo Pacchiano, Christoph Dann, Claudio Gentile

LogiGAN: Learning Logical Reasoning via Adversarial Pre-training

Xinyu Pi, Wanjun Zhong, Yan Gao, Nan Duan, Jian-Guang Lou

Learning Modular Simulations for Homogeneous Systems

Jayesh K. Gupta, Sai Vemprala, Ashish Kapoor

FocalNet: Focal Modulation Networks

Jianwei Yang, Chunyuan Li, Jianfeng Gao

Discovering Design Concepts for CAD Sketches

Yuezhi Yang, Hao Pan

FairVFL: A Fair Vertical Federated Learning Framework with Contrastive Adversarial Learning

Tao Qi, Fangzhao Wu, Chuhan Wu, Lingjuan Lyu, Tong Xu, Hao Liao, Zhongliang Yang, Yongfeng Huang, Xing Xie

Robustness Analysis of Video-Language Models Against Visual and Language Perturbations

Madeline C. Schiappa, Shruti Vyas, Hamid Palangi, Yogesh S. Rawat, Vibhav Vineet

Interaction-Grounded Learning with Action-Inclusive Feedback

Tengyang Xie, Akanksha Saran, Dylan J Foster, Lekan P Molu, Ida Momennejad, Nan Jiang, Paul Mineiro, John Langford

Expediting Large-Scale Vision Transformer for Dense Prediction without Fine-tuning

Weicong Liang, Yuhui Yuan, Henghui Ding, Xiao Luo, Weihong Lin, Ding Jia, Zheng Zhang, Chao Zhang, Han Hu

BinauralGrad: A Two-Stage Conditional Diffusion Probabilistic Model for Binaural Audio Synthesis

Yichong Leng, Zehua Chen, Junliang Guo, Haohe Liu, Jiawei Chen, Xu Tan, Danilo Mandic, Lei He, Xiangyang Li,, Tao Qin, Sheng Zhao, Tie-Yan Liu

CATER: Intellectual Property Protection on Text Generation APIs via Conditional Watermarks

Xuanli He, Qiongkai Xu, Yi Zeng, Lingjuan Lyu, Fangzhao Wu, Jiwei Li, Ruoxi Jia

Bootstrapped Transformer for Offline Reinforcement Learning

Kerong Wang, Hanye Zhao, Xufang Luo, Kan Ren, Weinan Zhang, Dongsheng Li

Maximizing and Satisficing in Multi-armed Bandits with Graph Information

Symbolic Distillation for Learned TCP Congestion Control

S P Sharan, Wenqing Zheng, Kuo-Feng Hsu, Jiarong Xing, Ang Chen, Zhangyang Wang

Diffusion Models as Plug-and-Play Priors

Alexandros Graikos, Nikolay Malkin, Nebojsa Jojic, Dimitris Samaras

Exploring evolution-based & -free protein language models as protein function predictors

Mingyang Hu, Fajie Yuan, Kevin Yang, Fusong Ju, Jin Sun, Hui Wang, Fei Yang, Qiuyang Ding

Recurrent Convolutional Neural Networks Learn Succinct Learning Algorithms

Surbhi Goel, Sham Kakade, Adam Tauman Kalai, Cyril Zhang

MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fields

Ilyes Batatia, Dávid Péter Kovács, Gregor Simm, Christoph Ortner, Gábor Csányi

Visual Clues: Bridging Vision and Language Foundations for Image Paragraph Captioning

Yujia Xie, Luowei Zhou, Xiyang Dai, Lu Yuan, Nguyen Bach, Ce Liu, Michael Zeng

Language Models with Image Descriptors are Strong Few-Shot Video-Language Learners

Zhenhailong Wang, Manling Li, Ruochen Xu, Luowei Zhou, Jie Lei, Xudong Lin, Shuohang Wang, Ziyi Yang, Chenguang Zhu, Derek Hoiem, Shih-Fu Chang, Mohit Bansal, Heng Ji

USB: A Unified Semi-supervised Learning Benchmark for Classification

Yidong Wang, Hao Chen, Yue Fan, Wang Sun, Ran Tao, Wenxin Hou, Renjie Wang, Linyi Yang, Zhi Zhou, Lan-Zhe Guo, Heli Qi, Zhen Wu, Yu-Feng Li, Satoshi Nakamura, Wei Ye, Marios Savvides, Bhiksha Raj, Takahiro Shinozaki, Bernt Schiele, Jindong Wang, Xing Xie, Yue Zhang

MoCapAct: A Multi-Task Dataset for Simulated Humanoid Control

Nolan Wagener, Andrey Kolobov, Felipe Vieira Frujeri, Ricky Loynd, Ching-An Cheng, Matthew Hausknecht

Reinforcement Learning with Automated Auxiliary Loss Search

Tairan He, Yuge Zhang, Kan Ren, Minghuan Liu, Che Wang, Weinan Zhang, Yuqing Yang, Dongsheng Li

A Neural Corpus Indexer for Document Retrieval

Yujing Wang, Yingyan Hou, Haonan Wang, Ziming Miao, Shibin Wu, Hao Sun, Qi Chen, Yuqing Xia, Chengmin Chi, Guoshuai Zhao, Zheng Liu, Xing Xie, Hao Allen Sun, Weiwei Deng, Qi Zhang, Mao Yang

Neuron with Steady Response Leads to Better Generalization

Qiang Fu, Lun Du, Haitao Mao, Xu Chen, Wei Fang, Shi Han, Dongmei Zhang

AutoDistil: Few-shot Task-agnostic Neural Architecture Search for Distilling Large Language Models

Dongkuan Xu, Subhabrata (Subho) Mukherjee, Xiaodong Liu, Debadeepta Dey, Wenhui Wang, Xiang Zhang, Ahmed Awadallah, Jianfeng Gao

Your Transformer May Not be as Powerful as You Expect

Shengjie Luo, Shanda Li, Shuxin Zheng, Tie-Yan Liu, Liwei Wang, Di He

Transcormer: Transformer for Sentence Scoring with Sliding Language Modeling

Kaitao Song, Yichong Leng, Xu Tan, Yicheng Zou, Tao Qin, Dongsheng Li

Partial Identification of Treatment Effects with Implicit Generative Models

Vahid Balazadeh, Vasilis Syrgkanis, Rahul Gopal Krishnan

Refined Dimension-Dependent Analysis for Private Convex Learning and Implications for Fine-Tuning

Xuechen Li, Daogao Liu, Tatsunori Hashimoto, Huseyin Inan, Janardhan (Jana) Kulkarni, YinTat Lee, Abhradeep Guha Thakurta

Why GANs are overkill for NLP

David Alvarez-Melis, Vikas Garg, Adam Tauman Kalai

Sparse Gaussian Process Hyperparameters: Optimize or Integrate?

Wessel Bruinsma

Learning General World Models in a Handful of Reward-Free Deployments

Yingchen Xu, Jack Parker-Holder, Aldo Pacchiano, Philip J. Ball, Oleh Rybkin, Stephen J. Roberts, Tim Rocktäschel

Non-Gaussian Tensor Programs

Eugene Golikov, Greg Yang

Rapid Model Architecture Adaption for Meta-Learning

Yiren Zhao, Xitong Gao, Ilia Shumailov, Nicolo Fusi, Robert D. Mullins

VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Experts

Hangbo Bao, Wenhui Wang, Li Dong, Qiang Liu, Owais Khan Mohammed, Kriti Aggarwal, Subhojit Som, Furu Wei

The Stability-Efficiency Dilemma: Investigating Sequence Length Warmup for Training GPT Models

Conglong Li, Minjia Zhang, Yuxiong He

Does Momentum Change the Implicit Regularization on Separable Data?

Bohan Wang, Qi Meng, Huishuai Zhang, Ruoyu Sun, Wei Chen, Zhi-Ming Ma, Tie-Yan Liu

NUWA-Infinity: Autoregressive over Autoregressive Generation for Infinite Visual Synthesis

Chenfei Wu, Jian Liang, Xiaowei Hu, Zhe Gan, Jianfeng Wang, Lijuan Wang, Zicheng Liu, Yuejian Fang, Nan Duan

Near-Optimal Correlation Clustering with Privacy

Vincent Cohen-Addad, Chenglin Fan, Silvio Lattanzi, Slobodan Mitrović, Ashkan Norouzi-Fard, Nikos Parotsidis, Jakub Tarnawski

High-dimensional Asymptotics of Feature Learning: How One Gradient Step Improves the Representation

Jimmy Ba, Murat A Erdogdu, Taiji Suzuki, Zhichao Wang, Denny Wu, Greg Yang

Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limit

Boaz Barak, Benjamin L. Edelman, Surbhi Goel, Sham M. Kakade, Eran Malach, Cyril Zhang

OpenFWI: Large-scale Multi-structural Benchmark Datasets for Full Waveform Inversion

Chengyuan Deng, Shihang Feng, Hanchen Wang, Xitong Zhang, Peng Jin, Yinan Feng, Qili Zeng, Yinpeng Chen, Youzuo Lin

ELEVATER: A Benchmark and Toolkit for Evaluating Language-Augmented Visual Models

Chunyuan Li, Haotian Liu, Liunian Harold Li, Pengchuan Zhang, Jyoti Aneja, Jianwei Yang, Ping Jin, Yong Jae Lee, Houdong Hu, Zicheng Liu, Jianfeng Gao

Two-Stream Network for Sign Language Recognition and Translation

Yutong Chen, Ronglai Zuo, Fangyun Wei, Yu Wu, Shujie Liu, Brian Mak

AniFaceGAN: Animatable 3D-Aware Face Image Generation for Video Avatars

Yue Wu, Yu Deng, Jiaolong Yang, Fangyun Wei, Qifeng Chen, Xin Tong

ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers

Zhewei Yao, Reza Yazdani Aminabadi, Minjia Zhang, Xiaoxia Wu, Conglong Li, Yuxiong He (yuxhe)

K-LITE: Learning Transferable Visual Models with External Knowledge

Sheng Shen, Chunyuan Li, Xiaowei Hu, Yujia Xie, Jianwei Yang, Pengchuan Zhang, Zhe Gan, Lijuan Wang, Lu Yuan, Ce Liu, Kurt Keutzer, Trevor Darrell, Anna Rohrbach, Jianfeng Gao

When does return-conditioned supervised learning work for offline reinforcement learning?

David Brandfonbrener, Alberto Bietti, Jacob Buckman, Romain Laroche, Joan Bruna

Riemannian Diffusion Models

Chin-Wei Huang, Milad Aghajohari, Avishek Joey Bose, Prakash Panangaden, Aaron Courville

Simultaneous Missing Value Imputation and Structure Learning with Groups

Pablo Morales-Alvarez, Wenbo Gong, Angus Lamb, Simon Woodhead, Simon Peyton Jones, Nick Pawlowski, Miltos Allamanis, Cheng Zhang

Repairing Neural Networks by Leaving the Right Past Behind

Ryutaro Tanno, Melanie Fernandez Pradier, Aditya Nori, Yingzhen Li

Batch-Size Independent Regret Bounds for Combinatorial Semi-Bandits with Probabilistically Triggered Arms or Independent Arms

Xutong Liu, Jinhang Zuo, Siwei Wang, Carlee Joe-Wong, John C.S. Lui, Wei Chen

Coarse-to-Fine Vision-Language Pre-training with Fusion in the Backbone

Zi-Yi Dou, Aishwarya Kamath, Zhe Gan, Pengchuan Zhang, Jianfeng Wang, Linjie Li, Zicheng Liu, Ce Liu, Yann LeCun, Nanyun Peng, Jianfeng Gao, Lijuan Wang

Nonlinear MCMC for Bayesian Machine Learning

James Vuckovic

Robust Generalized Method of Moments: A Finite Sample Viewpoint

Dhruv Rohatgi, Vasilis Syrgkanis

PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategies

Guocheng Qian, Yuchen Li, Houwen Peng, Jinjie Mai, Hasan Abed Al Kader Hammoud, Mohamed Elhoseiny, Bernard Ghanem

SCAMPS: Synthetics for Camera Measurement of Physiological Signals

Daniel McDuff, Jeremiah (Miah) Wander, Xin Liu, Brian L. Hill, Javier Hernandez, Jonathan Lester, Tadas Baltrusaitis

BigBIO: A Framework for Data-Centric Biomedical Natural Language Processing

Sid Kiblawi

Topic Segmentation in the Wild: Towards Segmentation of Semi-structured & Unstructured Chats

Reshmi Ghosh, Harjeet Singh Kajal, Sharanya Kamath, Dhuri Shrivastava, Samyadeep Basu, Soundararajan Srinivasan

CASPR: Customer Activity Sequence based Prediction and Representation

Pin-Jung Chen, Sahil Bhatnagar, Damian Konrad Kowalczyk, Mayank Shrivastava

Why GANs are overkill for NLP

David Alvarez-Melis, Vikas Garg, Adam Tauman Kalai

Probing Classifiers are Unreliable for Concept Removal and Detection

Abhinav Kumar, Chenhao Tan, Amit Sharma

Non-Gaussian Tensor Programs

Eugene Golikov, Greg Yang

Fault-Aware Neural Code Rankers

Jeevana Priya Inala, Chenglong Wang, Mei Yang, Andres Codas, Mark Encarnación, Shuvendu Lahiri, Madan Musuvathi, Jianfeng Gao

Gradient Estimation with Discrete Stein Operators

Jiaxin Shi, Yuhao Zhou, Jessica Hwang, Michalis Titsias, Lester Mackey

Could Giant Pre-trained Image Models Extract Universal Representations?

Yutong Lin, Ze Liu, Zheng Zhang, Han Hu, Nanning Zheng, Stephen Lin, Yue Cao

You Never Stop Dancing: Non-freezing Dance Generation via Bank-constrained Manifold Projection

Jiangxin Sun, Chunyu Wang, Huang Hu, Hanjiang Lai, Zhi Jin, Jian-Fang Hu

RTFormer: Efficient Design for Real-Time Semantic Segmentation with Transformer

Jian Wang, Chenhui Gou, Qiman Wu, Haocheng Feng, Junyu Han, Errui Ding, Jingdong Wang

Maximum Class Separation as Inductive Bias in One Matrix

Tejaswi Kasarla, Gertjan J. Burghouts, Max van Spengler, Elise van der Pol, Rita Cucchiara, Pascal Mettes

K-LITE: Learning Transferable Visual Models with External Knowledge

Sheng Shen, Chunyuan Li, Xiaowei Hu, Yujia Xie, Jianwei Yang, Pengchuan Zhang, Zhe Gan, Lijuan Wang, Lu Yuan, Ce Liu, Kurt Keutzer, Trevor Darrell, Anna Rohrbach, Jianfeng Gao

Extreme Compression for Pre-trained Transformers Made Simple and Efficient

Xiaoxia Wu, Zhewei Yao, Minjia Zhang, Conglong Li, Yuxiong He

Censored Quantile Regression Neural Networks for Distribution-Free Survival Analysis

Tim Pearce, Jong-Hyeon Jeong, Yichen Jia, Jun Zhu

Tiered Reinforcement Learning: Pessimism in the Face of Uncertainty and Constant Regret

Jiawei Huang, Li Zhao, Tao Qin, Wei Chen, Nan Jiang, Tie-Yan Liu

Does Momentum Change the Implicit Regularization on Separable Data?

Bohan Wang, Qi Meng, Huishuai Zhang, Ruoyu Sun, Wei Chen, Zhi-Ming Ma, Tie-Yan Liu

ELEVATER: A Benchmark and Toolkit for Evaluating Language-Augmented Visual Models

Chunyuan Li, Haotian Liu, Liunian Harold Li, Pengchuan Zhang, Jyoti Aneja, Jianwei Yang, Ping Jin, Yong Jae Lee, Houdong Hu, Zicheng Liu, Jianfeng Gao

AniFaceGAN: Animatable 3D-Aware Face Image Generation for Video Avatars

Yue Wu, Yu Deng, Jiaolong Yang, Fangyun Wei, Qifeng Chen, Xin Tong

BinauralGrad: A Two-Stage Conditional Diffusion Probabilistic Model for Binaural Audio Synthesis

Yichong Leng, Zehua Chen, Junliang Guo, Haohe Liu, Jiawei Chen, Xu Tan, Danilo Mandic, Lei He, Xiangyang Li,, Tao Qin, Sheng Zhao, Tie-Yan Liu

OmniVL: One Foundation Model for Image-Language and Video-Language Tasks

Junke Wang, Dongdong Chen, Zuxuan Wu, Chong Luo, Luowei Zhou, Yucheng Zhao, Yujia Xie, Ce Liu, Yu-Gang Jiang, Lu Yuan

The Stability-Efficiency Dilemma: Investigating Sequence Length Warmup for Training GPT Models

Conglong Li, Minjia Zhang, Yuxiong He

LiteTransformerSearch: Training-free On-device Search for Efficient Autoregressive Language Models

Mojan Javaheripi, Shital Shah, Subhabrata (Subho) Mukherjee, Tomasz L. Religa, Caio C. T. Mendes, Gustavo H. de Rosa, Sébastien Bubeck, Farinaz Koushanfar, Debadeepta Dey

FocalNet: Focal Modulation Networks

Jianwei Yang, Chunyuan Li, Jianfeng Gao

Visual Concepts Tokenization

Tao Yang, Yuwang Wang, Yan Lu, Nanning Zheng

Discovering Design Concepts for CAD Sketches

Yuezhi Yang, Hao Pan

Learning (Very) Simple Generative Models Is Hard

Sitan Chen, Jerry Li, Yuanzhi Li

Uni[MASK]: Unified Inference in Sequential Decision Problems

Micah Carroll, Orr Paradise, Jessy Lin, Raluca Georgescu, Mingfei Sun, Dave Bignell, Stephanie Milani, Katja Hofmann, Matthew Hausknecht, Anca Dragan, Sam Devlin

Museformer: Transformer with Fine- and Coarse-Grained Attention for Music Generation

Botao Yu, Peiling Lu, Rui Wang, Wei Hu, Xu Tan, Wei Ye, Shikun Zhang, Tao Qin, Tie-Yan Liu

Two-Stream Network for Sign Language Recognition and Translation

Yutong Chen, Ronglai Zuo, Fangyun Wei, Yu Wu, Shujie Liu, Brian Mak

ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers

Zhewei Yao, Reza Yazdani Aminabadi, Minjia Zhang, Xiaoxia Wu, Conglong Li, Yuxiong He (yuxhe)

Expediting Large-Scale Vision Transformer for Dense Prediction without Fine-tuning

Weicong Liang, Yuhui Yuan, Henghui Ding, Xiao Luo, Weihong Lin, Ding Jia, Zheng Zhang, Chao Zhang, Han Hu

Quantized Training of Gradient Boosted Decision Trees

Yu Shi, Guolin Ke, Zhuoming Chen, Shuxin Zheng, Tie-Yan Liu

On the Complexity of Adversarial Decision Making

Dylan J Foster, Alexander Rakhlin, Ayush Sekhari, Karthik Sridharan

Causal Modeling of Soil Processes for Improved Generalization

Somya Sharma, Swati Sharma, Andy Neal, Sara Malvar, Eduardo Rodrigues, John Crawford, Emre Kiciman, Ranveer Chandra

Industry-scale CO2 Flow Simulations with Model-Parallel Fourier Neural Operators

Philipp Witte, Russell Hewett, Ranveer Chandra

Adaptive Bias Correction for Improved Subseasonal Forecasting

Soukayna Mouatadid, Paulo Orenstein, Genevieve Flaspohler, Judah Cohen, Miruna Oprescu, Ernest Fraenkel, Lester Mackey

Surrogate Modeling for Methane Dispersion Simulations Using Fourier Neural Operator

Qie Zhang, Micro Milletari, Yagna Oruganti, Philipp Witte