FlexNN: Efficient and Adaptive DNN Inference on Memory-Constrained Edge Devices
Xiangyu Li, Yuanchun Li, Yuanzhe Li, Ting Cao, Yunxin Liu
International Conference on Mobile Computing and Networking (MobiCom ’24) | November 2024
Ting Cao
Principal Research Manager
Publications
Publications by Year
-
VPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Models
Yifei Liu, Jicheng Wen, Yang Wang, Shengyu Ye, Li Lyna Zhang, Ting Cao, Cheng Li, Mao Yang
November 2024
Ladder: Enabling Efficient Low-Precision Deep Learning Computing through Hardware-aware Tensor Transformation
Lei Wang, Lingxiao Ma, Shijie Cao, Quanlu Zhang, Jilong Xue, Yining Shi, Ningxin Zheng, Ziming Miao, Fan Yang, Ting Cao, Yuqing Yang, Mao Yang
OSDI 2024 | July 2024
Pre-gated MoE: An Algorithm-System Co-Design for Fast and Scalable Mixture-of-Expert Inference
Ranggi Hwang, Jianyu Wei, Shijie Cao, Changho Hwang, Xiaohu Tang, Ting Cao, Mao Yang
ISCA 2024 | July 2024
Microsoft Research Focus https://www.microsoft.com/en-us/research/blog/research-focus-week-of-july-15-2024/
Empowering In-Browser Deep Learning Inference on Edge Devices with Just-in-Time Kernel Optimizations
Fucheng Jia, Shiqi Jiang, Ting Cao, Wei Cui, Tianrui Xia, Xu Cao, Yuanchun Li, Qipeng Wang, Deyu Zhang, Ju Ren, Yunxin Liu, Lili Qiu, Mao Yang
the 22nd Annual International Conference on Mobile Systems, Applications and Services (MobiSys'24) | June 2024
LitePred: Transferable and Scalable Latency Prediction for Hardware-Aware Neural Architecture Search
Chengquan Feng, Li Lyna Zhang, Yuanchi Liu, Jiahang Xu, Chengruidong Zhang, Zhiyuan Wang, Ting Cao, Mao Yang, Haisheng Tan
NSDI 2024 | April 2024
PIM-DL: Expanding the Applicability of Commodity DRAM-PIMs for Deep Learning via Algorithm-System Co-Optimization
Cong Li, Zhe Zhou, Yang Wang, Fan Yang, Ting Cao, Mao Yang, Yun Liang, Guangyu Sun
29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS'24) | April 2024
ConvStencil: Transform Stencil Computation to Matrix Multiplication on Tensor Cores
Yuetao Chen, Kun Li, Yuhao Wang, Donglin Bai, Lei Wang, Lingxiao Ma, Liang Yuan, Yunquan Zhang, Ting Cao, Mao Yang
The 29th ACM SIGPLAN Annual Symposium on Principles and Practice of Parallel Programming (PPoPP ’24) | March 2024
Best paper award
-
ElasticViT: Conflict-aware Supernet Training for Deploying Fast Vision Transformer on Diverse Mobile Devices
Chen Tang, Li Lyna Zhang, Huiqiang Jiang, Jiahang Xu, Ting Cao, Quanlu Zhang, Yuqing Yang, Zhi Wang, Mao Yang
ICCV 2023 | October 2023
LUT-NN: Empower Efficient Neural Network Inference with Centroid Learning and Table Lookup
Xiaohu Tang, Yang Wang, Ting Cao, Li Lyna Zhang, Qi Chen, Deng Cai, Yunxin Liu, Mao Yang
The 29th Annual International Conference On Mobile Computing And Networking (MobiCom'23) | October 2023
Microsoft Research Focus https://www.microsoft.com/en-us/research/blog/research-focus-week-of-november-8-2023/
SpaceEvo: Hardware-Friendly Search Space Design for Efficient INT8 Inference
Xudong Wang, Li Lyna Zhang, Jiahang Xu, Quanlu Zhang, Yujing Wang, Yuqing Yang, Ningxin Zheng, Ting Cao, Mao Yang
ICCV 2023 | October 2023
Accurate and Structured Pruning for Efficient Automatic Speech Recognition
Huiqiang Jiang, Li Lyna Zhang, Yuang Li, Yu Wu, Shijie Cao, Ting Cao, Yuqing Yang, Jinyu Li, Mao Yang, Lili Qiu
INTERSPEECH 2023 | August 2023
Constraint-aware and Ranking-distilled Token Pruning for Efficient Transformer Inference
Junyan Li, Li Lyna Zhang, Jiahang Xu, Yujing Wang, Shaoguang Yan, Yunqing Xia, Yuqing Yang, Ting Cao, Hao Sun, Weiwei Deng, Qi Zhang, Mao Yang
KDD 2023 | August 2023
VSPIM: SRAM Processing-in-Memory DNN Acceleration via Vector-Scalar Operations
Chen Nie, Chenyu Tang, Jie Lin, Huan Hu, Chenyang Lv, Ting Cao, Weifeng Zhang, Li Jiang, Xiaoyao Liang, Weikang Qian, Yanan Sun, Zhezhi He
IEEE Transactions on Computers | June 2023
Boosting DNN Cold Inference on Devices
Rongjie Yi, Ting Cao, Ao Zhou, Xiao Ma, Shangguang Wang, Mengwei Xu
The 21st Annual International Conference on Mobile Systems, Applications and Services (MobiSys ’23) | June 2023
Efficient GPU Kernels for N:M-SPARSE Weights in Deep Learning
Bin Lin, Ningxin Zheng, Lei Wang, Shijie Cao, Lingxiao Ma, Quanlu Zhang, Yi Zhu, Ting Cao, Jilong Xue, Yuqing Yang, Fan Yang
Sixth Conference on Machine Learning and Systems (MLSys'23) | June 2023
NN-Stretch: Automatic Neural Network Branching for Parallel Inference on Heterogeneous Multi-Processors
Jianyu Wei, Ting Cao, Shijie Cao, Shiqi Jiang, Shaowei Fu, Mao Yang, Yanyong Zhang, Yunxin Liu
The 21st International Conference on Mobile Systems, Applications, and Services (MobiSys ’23) | June 2023
-
Hyperion: A Generic and Distributed Mobile Offloading Framework on OpenCL
Ziyan Fu, Ju Ren, Yunxin Liu, Ting Cao, Deyu Zhang, Yuezhi Zhou, Yaoxue Zhang
ACM Conference on Embedded Networked Sensor Systems (SenSys ’22) | November 2022
Turbo: Opportunistic Enhancement for Edge Video Analytics
Yan Lu, Shiqi Jiang, Ting Cao, Yuanchao Shu
The 20th ACM Conference on Embedded Networked Sensor Systems (SenSys ’22) | November 2022
MobiDepth: Real-Time Depth Estimation Using On-Device Dual Cameras
Jinrui Zhang, Huan Yang, Ju Ren, Deyu Zhang, Bangwen He, Yuanchun Li, Ting Cao, Yaoxue Zhang, Yunxin Liu
The 28th Annual International Conference On Mobile Computing And Networking (MobiCom'22) | October 2022
SwiftPruner: Reinforced Evolutionary Pruning for Efficient Ad Relevance
Li Lyna Zhang, Youkow Homma, Yujing Wang, Min Wu, Mao Yang, Ruofei Zhang, Ting Cao, Wei Shen
ACM International Conference on Information and Knowledge Management (CIKM'22) | October 2022
CoDL: Efficient CPU-GPU Co-execution for Deep Learning Inference on Mobile Devices
Fucheng Jia, Deyu Zhang, Ting Cao, Shiqi Jiang, Yunxin Liu, Ju Ren, Yaoxue Zhang
Proceedings of the 20th Annual International Conference on Mobile Systems, Applications and Services (MobiSys'22) | June 2022
Romou: Rapidly Generate High-Performance Tensor Kernels for Mobile GPUs
Rendong Liang, Ting Cao, Jicheng Wen, Manni Wang, Yang Wang, Jianhua Zou, Yunxin Liu
The 28th Annual International Conference On Mobile Computing And Networking (MobiCom'22) | February 2022
2023 Top 100 Open Source Achievement Award
-
nn-METER: Towards Accurate Latency Prediction of DNN Inference on Diverse Edge Devices
Li Lyna Zhang, Shihao Han, Jianyu Wei, Ningxin Zheng, Ting Cao, Yuqing Yang, Yunxin Liu
GetMobile: Mobile Computing and Communications | December 2021, Vol 25(4): pp. 19-23
SigMobile Research Highlight
Unified Holistic Memory Management Supporting Multiple Big Data Processing Frameworks over Hybrid Memories
Lei Chen, Jiacheng Zhao, Chenxi Wang, Ting Cao, John Zigman, Haris Volos, Onur Mutlu, Fang Lv, Xiaobing Feng, Guoqing Harry Xu, Huimin Cui
ACM Transactions on Computer Systems | November 2021, Vol 39(1-4): pp. 1-38
AsyMo: Scalable and Efficient Deep-Learning Inference on Asymmetric Mobile CPUs
Manni Wang, Shaohua Ding, Ting Cao, Yunxin Liu, Fengyuan Xu
The 27th Annual International Conference On Mobile Computing And Networking (MobiCom'21) | October 2021
nn-Meter: Towards Accurate Latency Prediction of Deep-Learning Model Inference on Diverse Edge Devices
Li Lyna Zhang, Shihao Han, Jianyu Wei, Ningxin Zheng, Ting Cao, Yuqing Yang, Yunxin Liu
The 19th ACM International Conference on Mobile Systems, Applications, and Services (MobiSys'21) | June 2021
Best Paper Award
To Bridge Neural Network Design and Real-World Performance: A Behaviour Study for Neural Networks
Xiaohu Tang, Shihao Han, Li Lyna Zhang, Ting Cao, Yunxin Liu
MLSys | April 2021
-
Profiling and optimizing deep learning inference on mobile GPUs
Shiqi Jiang, Lihao Ran, Ting Cao, Yusen Xu, Yunxin Liu
Proceedings of the 11th ACM SIGOPS Asia-Pacific Workshop on Systems (APSys '20) | August 2020
-
Panthera: Holistic Memory Management for Big Data Processing over Hybrid Memories
Chenxi Wang, Huimin Cui, Ting Cao, John Zigman, Haris Volos, Onur Mutlu, Fang Lv, Xiaobing Feng, Guoqing Harry Xu
the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI'19) | June 2019
-
Efficient Management for Hybrid Memory in Managed Language Runtime
Chenxi Wang, Ting Cao, John Zigman, Fang Lv, Yunquan Zhang, Xiaobing Feng
13th International Conference on Network and Parallel Computing (NPC) | September 2016
CGO’17 Student Research Competition
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9966)
Parallel Processing Systems for Big Data: A Survey
Yunquan Zhang, Ting Cao, Shigang Li, Xinhui Tian, Liang Yuan, Haipeng Jia, Athanasios V. Vasilakos
Proceedings of the IEEE | August 2016, Vol 104(11): pp. 2114-2136
Portable Performance on Asymmetric Multicore Processors
Ivan Jibaja, Ting Cao, Stephen M. Blackburn, Kathryn S. McKinley
Proceedings of the 2016 International Symposium on Code Generation and Optimization (CGO) | March 2016
Publications by Research Area
-
VPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Models
Yifei Liu, Jicheng Wen, Yang Wang, Shengyu Ye, Li Lyna Zhang, Ting Cao, Cheng Li, Mao Yang
November 2024
Empowering In-Browser Deep Learning Inference on Edge Devices with Just-in-Time Kernel Optimizations
Fucheng Jia, Shiqi Jiang, Ting Cao, Wei Cui, Tianrui Xia, Xu Cao, Yuanchun Li, Qipeng Wang, Deyu Zhang, Ju Ren, Yunxin Liu, Lili Qiu, Mao Yang
the 22nd Annual International Conference on Mobile Systems, Applications and Services (MobiSys'24) | June 2024
ElasticViT: Conflict-aware Supernet Training for Deploying Fast Vision Transformer on Diverse Mobile Devices
Chen Tang, Li Lyna Zhang, Huiqiang Jiang, Jiahang Xu, Ting Cao, Quanlu Zhang, Yuqing Yang, Zhi Wang, Mao Yang
ICCV 2023 | October 2023
SpaceEvo: Hardware-Friendly Search Space Design for Efficient INT8 Inference
Xudong Wang, Li Lyna Zhang, Jiahang Xu, Quanlu Zhang, Yujing Wang, Yuqing Yang, Ningxin Zheng, Ting Cao, Mao Yang
ICCV 2023 | October 2023
Accurate and Structured Pruning for Efficient Automatic Speech Recognition
Huiqiang Jiang, Li Lyna Zhang, Yuang Li, Yu Wu, Shijie Cao, Ting Cao, Yuqing Yang, Jinyu Li, Mao Yang, Lili Qiu
INTERSPEECH 2023 | August 2023
Constraint-aware and Ranking-distilled Token Pruning for Efficient Transformer Inference
Junyan Li, Li Lyna Zhang, Jiahang Xu, Yujing Wang, Shaoguang Yan, Yunqing Xia, Yuqing Yang, Ting Cao, Hao Sun, Weiwei Deng, Qi Zhang, Mao Yang
KDD 2023 | August 2023
Hyperion: A Generic and Distributed Mobile Offloading Framework on OpenCL
Ziyan Fu, Ju Ren, Yunxin Liu, Ting Cao, Deyu Zhang, Yuezhi Zhou, Yaoxue Zhang
ACM Conference on Embedded Networked Sensor Systems (SenSys ’22) | November 2022
Turbo: Opportunistic Enhancement for Edge Video Analytics
Yan Lu, Shiqi Jiang, Ting Cao, Yuanchao Shu
The 20th ACM Conference on Embedded Networked Sensor Systems (SenSys ’22) | November 2022
MobiDepth: Real-Time Depth Estimation Using On-Device Dual Cameras
Jinrui Zhang, Huan Yang, Ju Ren, Deyu Zhang, Bangwen He, Yuanchun Li, Ting Cao, Yaoxue Zhang, Yunxin Liu
The 28th Annual International Conference On Mobile Computing And Networking (MobiCom'22) | October 2022
SwiftPruner: Reinforced Evolutionary Pruning for Efficient Ad Relevance
Li Lyna Zhang, Youkow Homma, Yujing Wang, Min Wu, Mao Yang, Ruofei Zhang, Ting Cao, Wei Shen
ACM International Conference on Information and Knowledge Management (CIKM'22) | October 2022
CoDL: Efficient CPU-GPU Co-execution for Deep Learning Inference on Mobile Devices
Fucheng Jia, Deyu Zhang, Ting Cao, Shiqi Jiang, Yunxin Liu, Ju Ren, Yaoxue Zhang
Proceedings of the 20th Annual International Conference on Mobile Systems, Applications and Services (MobiSys'22) | June 2022
AsyMo: Scalable and Efficient Deep-Learning Inference on Asymmetric Mobile CPUs
Manni Wang, Shaohua Ding, Ting Cao, Yunxin Liu, Fengyuan Xu
The 27th Annual International Conference On Mobile Computing And Networking (MobiCom'21) | October 2021
nn-Meter: Towards Accurate Latency Prediction of Deep-Learning Model Inference on Diverse Edge Devices
Li Lyna Zhang, Shihao Han, Jianyu Wei, Ningxin Zheng, Ting Cao, Yuqing Yang, Yunxin Liu
The 19th ACM International Conference on Mobile Systems, Applications, and Services (MobiSys'21) | June 2021
Best Paper Award
To Bridge Neural Network Design and Real-World Performance: A Behaviour Study for Neural Networks
Xiaohu Tang, Shihao Han, Li Lyna Zhang, Ting Cao, Yunxin Liu
MLSys | April 2021
Profiling and optimizing deep learning inference on mobile GPUs
Shiqi Jiang, Lihao Ran, Ting Cao, Yusen Xu, Yunxin Liu
Proceedings of the 11th ACM SIGOPS Asia-Pacific Workshop on Systems (APSys '20) | August 2020
-
FlexNN: Efficient and Adaptive DNN Inference on Memory-Constrained Edge Devices
Xiangyu Li, Yuanchun Li, Yuanzhe Li, Ting Cao, Yunxin Liu
International Conference on Mobile Computing and Networking (MobiCom ’24) | November 2024
VPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Models
Yifei Liu, Jicheng Wen, Yang Wang, Shengyu Ye, Li Lyna Zhang, Ting Cao, Cheng Li, Mao Yang
November 2024
Ladder: Enabling Efficient Low-Precision Deep Learning Computing through Hardware-aware Tensor Transformation
Lei Wang, Lingxiao Ma, Shijie Cao, Quanlu Zhang, Jilong Xue, Yining Shi, Ningxin Zheng, Ziming Miao, Fan Yang, Ting Cao, Yuqing Yang, Mao Yang
OSDI 2024 | July 2024
Pre-gated MoE: An Algorithm-System Co-Design for Fast and Scalable Mixture-of-Expert Inference
Ranggi Hwang, Jianyu Wei, Shijie Cao, Changho Hwang, Xiaohu Tang, Ting Cao, Mao Yang
ISCA 2024 | July 2024
Microsoft Research Focus https://www.microsoft.com/en-us/research/blog/research-focus-week-of-july-15-2024/
Empowering In-Browser Deep Learning Inference on Edge Devices with Just-in-Time Kernel Optimizations
Fucheng Jia, Shiqi Jiang, Ting Cao, Wei Cui, Tianrui Xia, Xu Cao, Yuanchun Li, Qipeng Wang, Deyu Zhang, Ju Ren, Yunxin Liu, Lili Qiu, Mao Yang
the 22nd Annual International Conference on Mobile Systems, Applications and Services (MobiSys'24) | June 2024
LitePred: Transferable and Scalable Latency Prediction for Hardware-Aware Neural Architecture Search
Chengquan Feng, Li Lyna Zhang, Yuanchi Liu, Jiahang Xu, Chengruidong Zhang, Zhiyuan Wang, Ting Cao, Mao Yang, Haisheng Tan
NSDI 2024 | April 2024
PIM-DL: Expanding the Applicability of Commodity DRAM-PIMs for Deep Learning via Algorithm-System Co-Optimization
Cong Li, Zhe Zhou, Yang Wang, Fan Yang, Ting Cao, Mao Yang, Yun Liang, Guangyu Sun
29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS'24) | April 2024
ConvStencil: Transform Stencil Computation to Matrix Multiplication on Tensor Cores
Yuetao Chen, Kun Li, Yuhao Wang, Donglin Bai, Lei Wang, Lingxiao Ma, Liang Yuan, Yunquan Zhang, Ting Cao, Mao Yang
The 29th ACM SIGPLAN Annual Symposium on Principles and Practice of Parallel Programming (PPoPP ’24) | March 2024
Best paper award
LUT-NN: Empower Efficient Neural Network Inference with Centroid Learning and Table Lookup
Xiaohu Tang, Yang Wang, Ting Cao, Li Lyna Zhang, Qi Chen, Deng Cai, Yunxin Liu, Mao Yang
The 29th Annual International Conference On Mobile Computing And Networking (MobiCom'23) | October 2023
Microsoft Research Focus https://www.microsoft.com/en-us/research/blog/research-focus-week-of-november-8-2023/
ElasticViT: Conflict-aware Supernet Training for Deploying Fast Vision Transformer on Diverse Mobile Devices
Chen Tang, Li Lyna Zhang, Huiqiang Jiang, Jiahang Xu, Ting Cao, Quanlu Zhang, Yuqing Yang, Zhi Wang, Mao Yang
ICCV 2023 | October 2023
SpaceEvo: Hardware-Friendly Search Space Design for Efficient INT8 Inference
Xudong Wang, Li Lyna Zhang, Jiahang Xu, Quanlu Zhang, Yujing Wang, Yuqing Yang, Ningxin Zheng, Ting Cao, Mao Yang
ICCV 2023 | October 2023
Constraint-aware and Ranking-distilled Token Pruning for Efficient Transformer Inference
Junyan Li, Li Lyna Zhang, Jiahang Xu, Yujing Wang, Shaoguang Yan, Yunqing Xia, Yuqing Yang, Ting Cao, Hao Sun, Weiwei Deng, Qi Zhang, Mao Yang
KDD 2023 | August 2023
VSPIM: SRAM Processing-in-Memory DNN Acceleration via Vector-Scalar Operations
Chen Nie, Chenyu Tang, Jie Lin, Huan Hu, Chenyang Lv, Ting Cao, Weifeng Zhang, Li Jiang, Xiaoyao Liang, Weikang Qian, Yanan Sun, Zhezhi He
IEEE Transactions on Computers | June 2023
NN-Stretch: Automatic Neural Network Branching for Parallel Inference on Heterogeneous Multi-Processors
Jianyu Wei, Ting Cao, Shijie Cao, Shiqi Jiang, Shaowei Fu, Mao Yang, Yanyong Zhang, Yunxin Liu
The 21st International Conference on Mobile Systems, Applications, and Services (MobiSys ’23) | June 2023
Boosting DNN Cold Inference on Devices
Rongjie Yi, Ting Cao, Ao Zhou, Xiao Ma, Shangguang Wang, Mengwei Xu
The 21st Annual International Conference on Mobile Systems, Applications and Services (MobiSys ’23) | June 2023
Efficient GPU Kernels for N:M-SPARSE Weights in Deep Learning
Bin Lin, Ningxin Zheng, Lei Wang, Shijie Cao, Lingxiao Ma, Quanlu Zhang, Yi Zhu, Ting Cao, Jilong Xue, Yuqing Yang, Fan Yang
Sixth Conference on Machine Learning and Systems (MLSys'23) | June 2023
Hyperion: A Generic and Distributed Mobile Offloading Framework on OpenCL
Ziyan Fu, Ju Ren, Yunxin Liu, Ting Cao, Deyu Zhang, Yuezhi Zhou, Yaoxue Zhang
ACM Conference on Embedded Networked Sensor Systems (SenSys ’22) | November 2022
Turbo: Opportunistic Enhancement for Edge Video Analytics
Yan Lu, Shiqi Jiang, Ting Cao, Yuanchao Shu
The 20th ACM Conference on Embedded Networked Sensor Systems (SenSys ’22) | November 2022
MobiDepth: Real-Time Depth Estimation Using On-Device Dual Cameras
Jinrui Zhang, Huan Yang, Ju Ren, Deyu Zhang, Bangwen He, Yuanchun Li, Ting Cao, Yaoxue Zhang, Yunxin Liu
The 28th Annual International Conference On Mobile Computing And Networking (MobiCom'22) | October 2022
CoDL: Efficient CPU-GPU Co-execution for Deep Learning Inference on Mobile Devices
Fucheng Jia, Deyu Zhang, Ting Cao, Shiqi Jiang, Yunxin Liu, Ju Ren, Yaoxue Zhang
Proceedings of the 20th Annual International Conference on Mobile Systems, Applications and Services (MobiSys'22) | June 2022
Romou: Rapidly Generate High-Performance Tensor Kernels for Mobile GPUs
Rendong Liang, Ting Cao, Jicheng Wen, Manni Wang, Yang Wang, Jianhua Zou, Yunxin Liu
The 28th Annual International Conference On Mobile Computing And Networking (MobiCom'22) | February 2022
2023 Top 100 Open Source Achievement Award
nn-METER: Towards Accurate Latency Prediction of DNN Inference on Diverse Edge Devices
Li Lyna Zhang, Shihao Han, Jianyu Wei, Ningxin Zheng, Ting Cao, Yuqing Yang, Yunxin Liu
GetMobile: Mobile Computing and Communications | December 2021, Vol 25(4): pp. 19-23
SigMobile Research Highlight
Unified Holistic Memory Management Supporting Multiple Big Data Processing Frameworks over Hybrid Memories
Lei Chen, Jiacheng Zhao, Chenxi Wang, Ting Cao, John Zigman, Haris Volos, Onur Mutlu, Fang Lv, Xiaobing Feng, Guoqing Harry Xu, Huimin Cui
ACM Transactions on Computer Systems | November 2021, Vol 39(1-4): pp. 1-38
AsyMo: Scalable and Efficient Deep-Learning Inference on Asymmetric Mobile CPUs
Manni Wang, Shaohua Ding, Ting Cao, Yunxin Liu, Fengyuan Xu
The 27th Annual International Conference On Mobile Computing And Networking (MobiCom'21) | October 2021
nn-Meter: Towards Accurate Latency Prediction of Deep-Learning Model Inference on Diverse Edge Devices
Li Lyna Zhang, Shihao Han, Jianyu Wei, Ningxin Zheng, Ting Cao, Yuqing Yang, Yunxin Liu
The 19th ACM International Conference on Mobile Systems, Applications, and Services (MobiSys'21) | June 2021
Best Paper Award
Panthera: Holistic Memory Management for Big Data Processing over Hybrid Memories
Chenxi Wang, Huimin Cui, Ting Cao, John Zigman, Haris Volos, Onur Mutlu, Fang Lv, Xiaobing Feng, Guoqing Harry Xu
the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI'19) | June 2019
Parallel Processing Systems for Big Data: A Survey
Yunquan Zhang, Ting Cao, Shigang Li, Xinhui Tian, Liang Yuan, Haipeng Jia, Athanasios V. Vasilakos
Proceedings of the IEEE | August 2016, Vol 104(11): pp. 2114-2136
-
Accurate and Structured Pruning for Efficient Automatic Speech Recognition
Huiqiang Jiang, Li Lyna Zhang, Yuang Li, Yu Wu, Shijie Cao, Ting Cao, Yuqing Yang, Jinyu Li, Mao Yang, Lili Qiu
INTERSPEECH 2023 | August 2023
-
AsyMo: Scalable and Efficient Deep-Learning Inference on Asymmetric Mobile CPUs
Manni Wang, Shaohua Ding, Ting Cao, Yunxin Liu, Fengyuan Xu
The 27th Annual International Conference On Mobile Computing And Networking (MobiCom'21) | October 2021
nn-Meter: Towards Accurate Latency Prediction of Deep-Learning Model Inference on Diverse Edge Devices
Li Lyna Zhang, Shihao Han, Jianyu Wei, Ningxin Zheng, Ting Cao, Yuqing Yang, Yunxin Liu
The 19th ACM International Conference on Mobile Systems, Applications, and Services (MobiSys'21) | June 2021
Best Paper Award
To Bridge Neural Network Design and Real-World Performance: A Behaviour Study for Neural Networks
Xiaohu Tang, Shihao Han, Li Lyna Zhang, Ting Cao, Yunxin Liu
MLSys | April 2021
Profiling and optimizing deep learning inference on mobile GPUs
Shiqi Jiang, Lihao Ran, Ting Cao, Yusen Xu, Yunxin Liu
Proceedings of the 11th ACM SIGOPS Asia-Pacific Workshop on Systems (APSys '20) | August 2020
-
Efficient Management for Hybrid Memory in Managed Language Runtime
Chenxi Wang, Ting Cao, John Zigman, Fang Lv, Yunquan Zhang, Xiaobing Feng
13th International Conference on Network and Parallel Computing (NPC) | September 2016
CGO’17 Student Research Competition
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9966)
Portable Performance on Asymmetric Multicore Processors
Ivan Jibaja, Ting Cao, Stephen M. Blackburn, Kathryn S. McKinley
Proceedings of the 2016 International Symposium on Code Generation and Optimization (CGO) | March 2016
Publications by Type
-
FlexNN: Efficient and Adaptive DNN Inference on Memory-Constrained Edge Devices
Xiangyu Li, Yuanchun Li, Yuanzhe Li, Ting Cao, Yunxin Liu
International Conference on Mobile Computing and Networking (MobiCom ’24) | November 2024
VPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Models
Yifei Liu, Jicheng Wen, Yang Wang, Shengyu Ye, Li Lyna Zhang, Ting Cao, Cheng Li, Mao Yang
November 2024
Ladder: Enabling Efficient Low-Precision Deep Learning Computing through Hardware-aware Tensor Transformation
Lei Wang, Lingxiao Ma, Shijie Cao, Quanlu Zhang, Jilong Xue, Yining Shi, Ningxin Zheng, Ziming Miao, Fan Yang, Ting Cao, Yuqing Yang, Mao Yang
OSDI 2024 | July 2024
Pre-gated MoE: An Algorithm-System Co-Design for Fast and Scalable Mixture-of-Expert Inference
Ranggi Hwang, Jianyu Wei, Shijie Cao, Changho Hwang, Xiaohu Tang, Ting Cao, Mao Yang
ISCA 2024 | July 2024
Microsoft Research Focus https://www.microsoft.com/en-us/research/blog/research-focus-week-of-july-15-2024/
Empowering In-Browser Deep Learning Inference on Edge Devices with Just-in-Time Kernel Optimizations
Fucheng Jia, Shiqi Jiang, Ting Cao, Wei Cui, Tianrui Xia, Xu Cao, Yuanchun Li, Qipeng Wang, Deyu Zhang, Ju Ren, Yunxin Liu, Lili Qiu, Mao Yang
the 22nd Annual International Conference on Mobile Systems, Applications and Services (MobiSys'24) | June 2024
LitePred: Transferable and Scalable Latency Prediction for Hardware-Aware Neural Architecture Search
Chengquan Feng, Li Lyna Zhang, Yuanchi Liu, Jiahang Xu, Chengruidong Zhang, Zhiyuan Wang, Ting Cao, Mao Yang, Haisheng Tan
NSDI 2024 | April 2024
PIM-DL: Expanding the Applicability of Commodity DRAM-PIMs for Deep Learning via Algorithm-System Co-Optimization
Cong Li, Zhe Zhou, Yang Wang, Fan Yang, Ting Cao, Mao Yang, Yun Liang, Guangyu Sun
29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS'24) | April 2024
ConvStencil: Transform Stencil Computation to Matrix Multiplication on Tensor Cores
Yuetao Chen, Kun Li, Yuhao Wang, Donglin Bai, Lei Wang, Lingxiao Ma, Liang Yuan, Yunquan Zhang, Ting Cao, Mao Yang
The 29th ACM SIGPLAN Annual Symposium on Principles and Practice of Parallel Programming (PPoPP ’24) | March 2024
Best paper award
LUT-NN: Empower Efficient Neural Network Inference with Centroid Learning and Table Lookup
Xiaohu Tang, Yang Wang, Ting Cao, Li Lyna Zhang, Qi Chen, Deng Cai, Yunxin Liu, Mao Yang
The 29th Annual International Conference On Mobile Computing And Networking (MobiCom'23) | October 2023
Microsoft Research Focus https://www.microsoft.com/en-us/research/blog/research-focus-week-of-november-8-2023/
ElasticViT: Conflict-aware Supernet Training for Deploying Fast Vision Transformer on Diverse Mobile Devices
Chen Tang, Li Lyna Zhang, Huiqiang Jiang, Jiahang Xu, Ting Cao, Quanlu Zhang, Yuqing Yang, Zhi Wang, Mao Yang
ICCV 2023 | October 2023
SpaceEvo: Hardware-Friendly Search Space Design for Efficient INT8 Inference
Xudong Wang, Li Lyna Zhang, Jiahang Xu, Quanlu Zhang, Yujing Wang, Yuqing Yang, Ningxin Zheng, Ting Cao, Mao Yang
ICCV 2023 | October 2023
Accurate and Structured Pruning for Efficient Automatic Speech Recognition
Huiqiang Jiang, Li Lyna Zhang, Yuang Li, Yu Wu, Shijie Cao, Ting Cao, Yuqing Yang, Jinyu Li, Mao Yang, Lili Qiu
INTERSPEECH 2023 | August 2023
Constraint-aware and Ranking-distilled Token Pruning for Efficient Transformer Inference
Junyan Li, Li Lyna Zhang, Jiahang Xu, Yujing Wang, Shaoguang Yan, Yunqing Xia, Yuqing Yang, Ting Cao, Hao Sun, Weiwei Deng, Qi Zhang, Mao Yang
KDD 2023 | August 2023
NN-Stretch: Automatic Neural Network Branching for Parallel Inference on Heterogeneous Multi-Processors
Jianyu Wei, Ting Cao, Shijie Cao, Shiqi Jiang, Shaowei Fu, Mao Yang, Yanyong Zhang, Yunxin Liu
The 21st International Conference on Mobile Systems, Applications, and Services (MobiSys ’23) | June 2023
Boosting DNN Cold Inference on Devices
Rongjie Yi, Ting Cao, Ao Zhou, Xiao Ma, Shangguang Wang, Mengwei Xu
The 21st Annual International Conference on Mobile Systems, Applications and Services (MobiSys ’23) | June 2023
Efficient GPU Kernels for N:M-SPARSE Weights in Deep Learning
Bin Lin, Ningxin Zheng, Lei Wang, Shijie Cao, Lingxiao Ma, Quanlu Zhang, Yi Zhu, Ting Cao, Jilong Xue, Yuqing Yang, Fan Yang
Sixth Conference on Machine Learning and Systems (MLSys'23) | June 2023
Hyperion: A Generic and Distributed Mobile Offloading Framework on OpenCL
Ziyan Fu, Ju Ren, Yunxin Liu, Ting Cao, Deyu Zhang, Yuezhi Zhou, Yaoxue Zhang
ACM Conference on Embedded Networked Sensor Systems (SenSys ’22) | November 2022
Turbo: Opportunistic Enhancement for Edge Video Analytics
Yan Lu, Shiqi Jiang, Ting Cao, Yuanchao Shu
The 20th ACM Conference on Embedded Networked Sensor Systems (SenSys ’22) | November 2022
MobiDepth: Real-Time Depth Estimation Using On-Device Dual Cameras
Jinrui Zhang, Huan Yang, Ju Ren, Deyu Zhang, Bangwen He, Yuanchun Li, Ting Cao, Yaoxue Zhang, Yunxin Liu
The 28th Annual International Conference On Mobile Computing And Networking (MobiCom'22) | October 2022
SwiftPruner: Reinforced Evolutionary Pruning for Efficient Ad Relevance
Li Lyna Zhang, Youkow Homma, Yujing Wang, Min Wu, Mao Yang, Ruofei Zhang, Ting Cao, Wei Shen
ACM International Conference on Information and Knowledge Management (CIKM'22) | October 2022
CoDL: Efficient CPU-GPU Co-execution for Deep Learning Inference on Mobile Devices
Fucheng Jia, Deyu Zhang, Ting Cao, Shiqi Jiang, Yunxin Liu, Ju Ren, Yaoxue Zhang
Proceedings of the 20th Annual International Conference on Mobile Systems, Applications and Services (MobiSys'22) | June 2022
Romou: Rapidly Generate High-Performance Tensor Kernels for Mobile GPUs
Rendong Liang, Ting Cao, Jicheng Wen, Manni Wang, Yang Wang, Jianhua Zou, Yunxin Liu
The 28th Annual International Conference On Mobile Computing And Networking (MobiCom'22) | February 2022
2023 Top 100 Open Source Achievement Award
AsyMo: Scalable and Efficient Deep-Learning Inference on Asymmetric Mobile CPUs
Manni Wang, Shaohua Ding, Ting Cao, Yunxin Liu, Fengyuan Xu
The 27th Annual International Conference On Mobile Computing And Networking (MobiCom'21) | October 2021
nn-Meter: Towards Accurate Latency Prediction of Deep-Learning Model Inference on Diverse Edge Devices
Li Lyna Zhang, Shihao Han, Jianyu Wei, Ningxin Zheng, Ting Cao, Yuqing Yang, Yunxin Liu
The 19th ACM International Conference on Mobile Systems, Applications, and Services (MobiSys'21) | June 2021
Best Paper Award
To Bridge Neural Network Design and Real-World Performance: A Behaviour Study for Neural Networks
Xiaohu Tang, Shihao Han, Li Lyna Zhang, Ting Cao, Yunxin Liu
MLSys | April 2021
Profiling and optimizing deep learning inference on mobile GPUs
Shiqi Jiang, Lihao Ran, Ting Cao, Yusen Xu, Yunxin Liu
Proceedings of the 11th ACM SIGOPS Asia-Pacific Workshop on Systems (APSys '20) | August 2020
Panthera: Holistic Memory Management for Big Data Processing over Hybrid Memories
Chenxi Wang, Huimin Cui, Ting Cao, John Zigman, Haris Volos, Onur Mutlu, Fang Lv, Xiaobing Feng, Guoqing Harry Xu
the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI'19) | June 2019
Efficient Management for Hybrid Memory in Managed Language Runtime
Chenxi Wang, Ting Cao, John Zigman, Fang Lv, Yunquan Zhang, Xiaobing Feng
13th International Conference on Network and Parallel Computing (NPC) | September 2016
CGO’17 Student Research Competition
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9966)
Portable Performance on Asymmetric Multicore Processors
Ivan Jibaja, Ting Cao, Stephen M. Blackburn, Kathryn S. McKinley
Proceedings of the 2016 International Symposium on Code Generation and Optimization (CGO) | March 2016
-
VSPIM: SRAM Processing-in-Memory DNN Acceleration via Vector-Scalar Operations
Chen Nie, Chenyu Tang, Jie Lin, Huan Hu, Chenyang Lv, Ting Cao, Weifeng Zhang, Li Jiang, Xiaoyao Liang, Weikang Qian, Yanan Sun, Zhezhi He
IEEE Transactions on Computers | June 2023
nn-METER: Towards Accurate Latency Prediction of DNN Inference on Diverse Edge Devices
Li Lyna Zhang, Shihao Han, Jianyu Wei, Ningxin Zheng, Ting Cao, Yuqing Yang, Yunxin Liu
GetMobile: Mobile Computing and Communications | December 2021, Vol 25(4): pp. 19-23
SigMobile Research Highlight
Unified Holistic Memory Management Supporting Multiple Big Data Processing Frameworks over Hybrid Memories
Lei Chen, Jiacheng Zhao, Chenxi Wang, Ting Cao, John Zigman, Haris Volos, Onur Mutlu, Fang Lv, Xiaobing Feng, Guoqing Harry Xu, Huimin Cui
ACM Transactions on Computer Systems | November 2021, Vol 39(1-4): pp. 1-38
Parallel Processing Systems for Big Data: A Survey
Yunquan Zhang, Ting Cao, Shigang Li, Xinhui Tian, Liang Yuan, Haipeng Jia, Athanasios V. Vasilakos
Proceedings of the IEEE | August 2016, Vol 104(11): pp. 2114-2136