Vision and Language Intelligence

VLC: Training Vision-Language Transformers from Captions

Liangke Gui, Qiuyuan Huang, Alex Hauptmann, Yonatan Bisk, Jianfeng Gao

Proceedings of Transactions on Machine Learning Research | May 2022

Publication Project

NICE: Neural Image Commenting Evaluation with an Emphasis on Emotion and Empathy

Kezhen Chen, Qiuyuan Huang, Daniel McDuff, Jianfeng Wang, Hamid Palangi, Xiang Gao, Kevin Li, Kenneth Forbus, Jianfeng Gao

NeurIPS 2020, Contributed talk in HLDS workshop. | October 2020

Publication Publication

Vision-Language Navigation Policy Learning and Adaptation

Xin Wang, Qiuyuan Huang, Asli Celikyilmaz, Jianfeng Gao, Dinghan Shen, Yuan-fang Wang, William Wang, Lei Zhang

IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2020 | February 2020

Publication Publication Project

TIGEr: Text-to-Image Grounding for Image Caption Evaluation

Ming Jiang, Qiuyuan Huang, Lei Zhang, Xin Wang, Pengchuan Zhang, Zhe Gan, Jana Diesner, Jianfeng Gao

EMNLP 2019 | November 2019

Publication

REO-Relevance, Extraness, Omission: A Fine-grained Evaluation for Image Captioning

Ming Jiang, Junjie Hu, Qiuyuan Huang, Lei Zhang, Jana Diesner, Jianfeng Gao

EMNLP 2019 | November 2019

Publication

Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation

Xin Wang, Qiuyuan Huang, Asli Celikyilmaz, Jianfeng Gao, Dinghan Shen, Yuan-Fang Weng, William Yang Wang, Lei Zhang

CVPR 2019 | June 2019

Best Student Paper Award in CVPR 2019.

Publication Video Project Project

Hierarchically Structured Reinforcement Learning for Topically Coherent Visual Story Generation

Qiuyuan Huang, Zhe Gan, Asli Celikyilmaz, Oliver Wu, Jianfeng Wang, Xiaodong He

AAAI 2019 | January 2019

Publication Project Project

Turbo Learning for CaptionBot and DrawingBot

Qiuyuan Huang, Pengchuan Zhang, Oliver Wu, Lei Zhang

NeurIPS 2018 | December 2018

PDF Publication Project

Stacked Attention Networks for Image Question Answering

Zichao Yang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Smola

CVPR 2016 | June 2016

Publication Publication

VLC: Training Vision-Language Transformers from Captions

Liangke Gui, Qiuyuan Huang, Alex Hauptmann, Yonatan Bisk, Jianfeng Gao

Proceedings of Transactions on Machine Learning Research | May 2022

Publication Project

NICE: Neural Image Commenting Evaluation with an Emphasis on Emotion and Empathy

Kezhen Chen, Qiuyuan Huang, Daniel McDuff, Jianfeng Wang, Hamid Palangi, Xiang Gao, Kevin Li, Kenneth Forbus, Jianfeng Gao

NeurIPS 2020, Contributed talk in HLDS workshop. | October 2020

Publication Publication

Vision-Language Navigation Policy Learning and Adaptation

Xin Wang, Qiuyuan Huang, Asli Celikyilmaz, Jianfeng Gao, Dinghan Shen, Yuan-fang Wang, William Wang, Lei Zhang

IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2020 | February 2020

Publication Publication Project

Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation

Xin Wang, Qiuyuan Huang, Asli Celikyilmaz, Jianfeng Gao, Dinghan Shen, Yuan-Fang Weng, William Yang Wang, Lei Zhang

CVPR 2019 | June 2019

Best Student Paper Award in CVPR 2019.

Publication Video Project Project

Hierarchically Structured Reinforcement Learning for Topically Coherent Visual Story Generation

Qiuyuan Huang, Zhe Gan, Asli Celikyilmaz, Oliver Wu, Jianfeng Wang, Xiaodong He

AAAI 2019 | January 2019

Publication Project Project

Turbo Learning for CaptionBot and DrawingBot

Qiuyuan Huang, Pengchuan Zhang, Oliver Wu, Lei Zhang

NeurIPS 2018 | December 2018

PDF Publication Project

Stacked Attention Networks for Image Question Answering

Zichao Yang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Smola

CVPR 2016 | June 2016

Publication Publication

VLC: Training Vision-Language Transformers from Captions

Liangke Gui, Qiuyuan Huang, Alex Hauptmann, Yonatan Bisk, Jianfeng Gao

Proceedings of Transactions on Machine Learning Research | May 2022

Publication Project

Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation

Xin Wang, Qiuyuan Huang, Asli Celikyilmaz, Jianfeng Gao, Dinghan Shen, Yuan-Fang Weng, William Yang Wang, Lei Zhang

CVPR 2019 | June 2019

Best Student Paper Award in CVPR 2019.

Publication Video Project Project

Hierarchically Structured Reinforcement Learning for Topically Coherent Visual Story Generation

Qiuyuan Huang, Zhe Gan, Asli Celikyilmaz, Oliver Wu, Jianfeng Wang, Xiaodong He

AAAI 2019 | January 2019

Publication Project Project

Turbo Learning for CaptionBot and DrawingBot

Qiuyuan Huang, Pengchuan Zhang, Oliver Wu, Lei Zhang

NeurIPS 2018 | December 2018

PDF Publication Project

VLC: Training Vision-Language Transformers from Captions

Liangke Gui, Qiuyuan Huang, Alex Hauptmann, Yonatan Bisk, Jianfeng Gao

Proceedings of Transactions on Machine Learning Research | May 2022

Publication Project

NICE: Neural Image Commenting Evaluation with an Emphasis on Emotion and Empathy

Kezhen Chen, Qiuyuan Huang, Daniel McDuff, Jianfeng Wang, Hamid Palangi, Xiang Gao, Kevin Li, Kenneth Forbus, Jianfeng Gao

NeurIPS 2020, Contributed talk in HLDS workshop. | October 2020

Publication Publication

TIGEr: Text-to-Image Grounding for Image Caption Evaluation

Ming Jiang, Qiuyuan Huang, Lei Zhang, Xin Wang, Pengchuan Zhang, Zhe Gan, Jana Diesner, Jianfeng Gao

EMNLP 2019 | November 2019

Publication

REO-Relevance, Extraness, Omission: A Fine-grained Evaluation for Image Captioning

Ming Jiang, Junjie Hu, Qiuyuan Huang, Lei Zhang, Jana Diesner, Jianfeng Gao

EMNLP 2019 | November 2019

Publication

Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation

Xin Wang, Qiuyuan Huang, Asli Celikyilmaz, Jianfeng Gao, Dinghan Shen, Yuan-Fang Weng, William Yang Wang, Lei Zhang

CVPR 2019 | June 2019

Best Student Paper Award in CVPR 2019.

Publication Video Project Project

Hierarchically Structured Reinforcement Learning for Topically Coherent Visual Story Generation

Qiuyuan Huang, Zhe Gan, Asli Celikyilmaz, Oliver Wu, Jianfeng Wang, Xiaodong He

AAAI 2019 | January 2019

Publication Project Project

Turbo Learning for CaptionBot and DrawingBot

Qiuyuan Huang, Pengchuan Zhang, Oliver Wu, Lei Zhang

NeurIPS 2018 | December 2018

PDF Publication Project

NICE: Neural Image Commenting Evaluation with an Emphasis on Emotion and Empathy

Kezhen Chen, Qiuyuan Huang, Daniel McDuff, Jianfeng Wang, Hamid Palangi, Xiang Gao, Kevin Li, Kenneth Forbus, Jianfeng Gao

NeurIPS 2020, Contributed talk in HLDS workshop. | October 2020

Publication Publication

Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation

Xin Wang, Qiuyuan Huang, Asli Celikyilmaz, Jianfeng Gao, Dinghan Shen, Yuan-Fang Weng, William Yang Wang, Lei Zhang

CVPR 2019 | June 2019

Best Student Paper Award in CVPR 2019.

Publication Video Project Project

Turbo Learning for CaptionBot and DrawingBot

Qiuyuan Huang, Pengchuan Zhang, Oliver Wu, Lei Zhang

NeurIPS 2018 | December 2018

PDF Publication Project

Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation

Xin Wang, Qiuyuan Huang, Asli Celikyilmaz, Jianfeng Gao, Dinghan Shen, Yuan-Fang Weng, William Yang Wang, Lei Zhang

CVPR 2019 | June 2019

Best Student Paper Award in CVPR 2019.

Publication Video Project Project

VLC: Training Vision-Language Transformers from Captions

Liangke Gui, Qiuyuan Huang, Alex Hauptmann, Yonatan Bisk, Jianfeng Gao

Proceedings of Transactions on Machine Learning Research | May 2022

Publication Project

Vision-Language Navigation Policy Learning and Adaptation

Xin Wang, Qiuyuan Huang, Asli Celikyilmaz, Jianfeng Gao, Dinghan Shen, Yuan-fang Wang, William Wang, Lei Zhang

IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2020 | February 2020

Publication Publication Project

NICE: Neural Image Commenting Evaluation with an Emphasis on Emotion and Empathy

Kezhen Chen, Qiuyuan Huang, Daniel McDuff, Jianfeng Wang, Hamid Palangi, Xiang Gao, Kevin Li, Kenneth Forbus, Jianfeng Gao

NeurIPS 2020, Contributed talk in HLDS workshop. | October 2020

Publication Publication

TIGEr: Text-to-Image Grounding for Image Caption Evaluation

Ming Jiang, Qiuyuan Huang, Lei Zhang, Xin Wang, Pengchuan Zhang, Zhe Gan, Jana Diesner, Jianfeng Gao

EMNLP 2019 | November 2019

Publication

REO-Relevance, Extraness, Omission: A Fine-grained Evaluation for Image Captioning

Ming Jiang, Junjie Hu, Qiuyuan Huang, Lei Zhang, Jana Diesner, Jianfeng Gao

EMNLP 2019 | November 2019

Publication

Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation

Xin Wang, Qiuyuan Huang, Asli Celikyilmaz, Jianfeng Gao, Dinghan Shen, Yuan-Fang Weng, William Yang Wang, Lei Zhang

CVPR 2019 | June 2019

Best Student Paper Award in CVPR 2019.

Publication Video Project Project

Hierarchically Structured Reinforcement Learning for Topically Coherent Visual Story Generation

Qiuyuan Huang, Zhe Gan, Asli Celikyilmaz, Oliver Wu, Jianfeng Wang, Xiaodong He

AAAI 2019 | January 2019

Publication Project Project

Turbo Learning for CaptionBot and DrawingBot

Qiuyuan Huang, Pengchuan Zhang, Oliver Wu, Lei Zhang

NeurIPS 2018 | December 2018

PDF Publication Project

Stacked Attention Networks for Image Question Answering

Zichao Yang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Smola

CVPR 2016 | June 2016

Publication Publication