Taming Throughout-Latency Tradeoff in LLM Inference with Sarathi-Serve
Amey Agrawal, Nitin Kedia, Ashish Panwar, Jayashree Mohan, Nipun Kwatra, Bhargav S. Gulavani, Alexey Tumanov, Ramachandran Ramjee
OSDI 2024 | May 2024
Nipun Kwatra
Principal Researcher
Publications
Publications by Year
-
VIDUR: A Large-Scale Simulation Framework for LLM Inference
Amey Agrawal, Nitin Kedia, Jayashree Mohan, Ashish Panwar, Nipun Kwatra, Bhargav S. Gulavani, Ramachandran Ramjee, Alexey Tumanov
The Seventh Annual Conference on Machine Learning and Systems | May 2024
Just-In-Time Checkpointing: Low Cost Error Recovery from Deep Learning Training Failures
Tanmaey Gupta, Sanjeev Krishnan, Rituraj Kumar, Abhishek Vijeev, Bhargav Gulavani, Nipun Kwatra, Ramachandran Ramjee, Muthian Sivathanu
2024 European Conference on Computer Systems | April 2024
-
SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills
Amey Agrawal, Ashish Panwar, Jayashree Mohan, Nipun Kwatra, Bhargav S. Gulavani, Ramachandran Ramjee
September 2023
-
Towards Automating Retinoscopy for Refractive Error Diagnosis
Aditya Aggarwal, Siddhartha Gairola, Uddeshya Upadhyay, Akshay P Vasishta, Diwakar Rao, Aditya Goyal, Kaushik Murali, Nipun Kwatra, Mohit Jain
August 2022
Keratoconus Classifier for Smartphone-based Corneal Topographer
Siddhartha Gairola, Pallavi Joshi, Anand Balasubramaniam, Kaushik Murali, Nipun Kwatra, Mohit Jain
IEEE International Engineering in Medicine and Biology Conference (EMBC) | July 2022
Varuna: Scalable, Low-cost Training of Massive Deep Learning Models
Sanjith Athlur, Nitika Saran, Muthian Sivathanu, Ramachandran Ramjee, Nipun Kwatra
EuroSys 2022 | April 2022
Best Paper Award
Singularity: Planet-Scale, Preemptive and Elastic Scheduling of AI Workloads
Dharma Shukla, Muthian Sivathanu, Srinidhi Viswanatha, Bhargav Gulavani, Rimma Nehme, Amey Agrawal, Chen Chen, Nipun Kwatra, Ramachandran Ramjee, Pankaj Sharma, Atul Katiyar, Vipul Modi, Vaibhav Sharma, Abhishek Singh, Shreshth Singhal, Kaustubh Welankar, Lu Xun, Ravi Anupindi, Karthik Elangovan, Hasibur Rehman, Zhou Lin, Rahul Seetharaman, Cheng Xu, Eddie Ailijiang, Suresh Krishnappa, Mark Russinovich
February 2022
-
RespireNet: A Deep Neural Network for Accurately Detecting Abnormal Lung Sounds in Limited Data Setting
Siddhartha Gairola, Francis Tom, Nipun Kwatra, Mohit Jain
IEEE International Engineering in Medicine and Biology Conference (EMBC) | October 2021
SmartKC: Smartphone-based Corneal Topographer for Keratoconus Detection
Siddhartha Gairola, Murtuza Bohra, Nadeem Shaheer, Navya Jayaprakash, Pallavi Joshi, Anand Balasubramaniam, Kaushik Murali, Nipun Kwatra, Mohit Jain
IMWUT | October 2021, Vol 5(4)
-
Unsupervised Clustering using Pseudo-semi-supervised Learning
Divam Gupta, Ramachandran Ramjee, Nipun Kwatra, Muthian Sivathanu
Eighth International Conference on Learning Representations (ICLR) | April 2020
Balancing efficiency and fairness in heterogeneous GPU clusters for deep learning
Shubham Chaudhary, Ramachandran Ramjee, Muthian Sivathanu, Nipun Kwatra, Srinidhi Viswanatha
Fifteenth European Conference on Computer Systems (EuroSys'20) | April 2020
-
Gandiva: Introspective Cluster Scheduling for Deep Learning
Wencong Xiao, Romil Bhardwaj, Ramachandran Ramjee, Muthian Sivathanu, Nipun Kwatra, Zhenhua Han, Pratyush Patel, Xuan Peng, Hanyu Zhao, Quanlu Zhang, Fan Yang, Lidong Zhou
13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2018) | October 2018
Publications by Research Area
-
Taming Throughout-Latency Tradeoff in LLM Inference with Sarathi-Serve
Amey Agrawal, Nitin Kedia, Ashish Panwar, Jayashree Mohan, Nipun Kwatra, Bhargav S. Gulavani, Alexey Tumanov, Ramachandran Ramjee
OSDI 2024 | May 2024
VIDUR: A Large-Scale Simulation Framework for LLM Inference
Amey Agrawal, Nitin Kedia, Jayashree Mohan, Ashish Panwar, Nipun Kwatra, Bhargav S. Gulavani, Ramachandran Ramjee, Alexey Tumanov
The Seventh Annual Conference on Machine Learning and Systems | May 2024
Just-In-Time Checkpointing: Low Cost Error Recovery from Deep Learning Training Failures
Tanmaey Gupta, Sanjeev Krishnan, Rituraj Kumar, Abhishek Vijeev, Bhargav Gulavani, Nipun Kwatra, Ramachandran Ramjee, Muthian Sivathanu
2024 European Conference on Computer Systems | April 2024
SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills
Amey Agrawal, Ashish Panwar, Jayashree Mohan, Nipun Kwatra, Bhargav S. Gulavani, Ramachandran Ramjee
September 2023
Varuna: Scalable, Low-cost Training of Massive Deep Learning Models
Sanjith Athlur, Nitika Saran, Muthian Sivathanu, Ramachandran Ramjee, Nipun Kwatra
EuroSys 2022 | April 2022
Best Paper Award
Singularity: Planet-Scale, Preemptive and Elastic Scheduling of AI Workloads
Dharma Shukla, Muthian Sivathanu, Srinidhi Viswanatha, Bhargav Gulavani, Rimma Nehme, Amey Agrawal, Chen Chen, Nipun Kwatra, Ramachandran Ramjee, Pankaj Sharma, Atul Katiyar, Vipul Modi, Vaibhav Sharma, Abhishek Singh, Shreshth Singhal, Kaustubh Welankar, Lu Xun, Ravi Anupindi, Karthik Elangovan, Hasibur Rehman, Zhou Lin, Rahul Seetharaman, Cheng Xu, Eddie Ailijiang, Suresh Krishnappa, Mark Russinovich
February 2022
Balancing efficiency and fairness in heterogeneous GPU clusters for deep learning
Shubham Chaudhary, Ramachandran Ramjee, Muthian Sivathanu, Nipun Kwatra, Srinidhi Viswanatha
Fifteenth European Conference on Computer Systems (EuroSys'20) | April 2020
Gandiva: Introspective Cluster Scheduling for Deep Learning
Wencong Xiao, Romil Bhardwaj, Ramachandran Ramjee, Muthian Sivathanu, Nipun Kwatra, Zhenhua Han, Pratyush Patel, Xuan Peng, Hanyu Zhao, Quanlu Zhang, Fan Yang, Lidong Zhou
13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2018) | October 2018
-
Towards Automating Retinoscopy for Refractive Error Diagnosis
Aditya Aggarwal, Siddhartha Gairola, Uddeshya Upadhyay, Akshay P Vasishta, Diwakar Rao, Aditya Goyal, Kaushik Murali, Nipun Kwatra, Mohit Jain
August 2022
Keratoconus Classifier for Smartphone-based Corneal Topographer
Siddhartha Gairola, Pallavi Joshi, Anand Balasubramaniam, Kaushik Murali, Nipun Kwatra, Mohit Jain
IEEE International Engineering in Medicine and Biology Conference (EMBC) | July 2022
RespireNet: A Deep Neural Network for Accurately Detecting Abnormal Lung Sounds in Limited Data Setting
Siddhartha Gairola, Francis Tom, Nipun Kwatra, Mohit Jain
IEEE International Engineering in Medicine and Biology Conference (EMBC) | October 2021
SmartKC: Smartphone-based Corneal Topographer for Keratoconus Detection
Siddhartha Gairola, Murtuza Bohra, Nadeem Shaheer, Navya Jayaprakash, Pallavi Joshi, Anand Balasubramaniam, Kaushik Murali, Nipun Kwatra, Mohit Jain
IMWUT | October 2021, Vol 5(4)
-
Keratoconus Classifier for Smartphone-based Corneal Topographer
Siddhartha Gairola, Pallavi Joshi, Anand Balasubramaniam, Kaushik Murali, Nipun Kwatra, Mohit Jain
IEEE International Engineering in Medicine and Biology Conference (EMBC) | July 2022
RespireNet: A Deep Neural Network for Accurately Detecting Abnormal Lung Sounds in Limited Data Setting
Siddhartha Gairola, Francis Tom, Nipun Kwatra, Mohit Jain
IEEE International Engineering in Medicine and Biology Conference (EMBC) | October 2021
Unsupervised Clustering using Pseudo-semi-supervised Learning
Divam Gupta, Ramachandran Ramjee, Nipun Kwatra, Muthian Sivathanu
Eighth International Conference on Learning Representations (ICLR) | April 2020
-
Keratoconus Classifier for Smartphone-based Corneal Topographer
Siddhartha Gairola, Pallavi Joshi, Anand Balasubramaniam, Kaushik Murali, Nipun Kwatra, Mohit Jain
IEEE International Engineering in Medicine and Biology Conference (EMBC) | July 2022
RespireNet: A Deep Neural Network for Accurately Detecting Abnormal Lung Sounds in Limited Data Setting
Siddhartha Gairola, Francis Tom, Nipun Kwatra, Mohit Jain
IEEE International Engineering in Medicine and Biology Conference (EMBC) | October 2021
SmartKC: Smartphone-based Corneal Topographer for Keratoconus Detection
Siddhartha Gairola, Murtuza Bohra, Nadeem Shaheer, Navya Jayaprakash, Pallavi Joshi, Anand Balasubramaniam, Kaushik Murali, Nipun Kwatra, Mohit Jain
IMWUT | October 2021, Vol 5(4)
-
SmartKC: Smartphone-based Corneal Topographer for Keratoconus Detection
Siddhartha Gairola, Murtuza Bohra, Nadeem Shaheer, Navya Jayaprakash, Pallavi Joshi, Anand Balasubramaniam, Kaushik Murali, Nipun Kwatra, Mohit Jain
IMWUT | October 2021, Vol 5(4)
-
RespireNet: A Deep Neural Network for Accurately Detecting Abnormal Lung Sounds in Limited Data Setting
Siddhartha Gairola, Francis Tom, Nipun Kwatra, Mohit Jain
IEEE International Engineering in Medicine and Biology Conference (EMBC) | October 2021
Publications by Type
-
Taming Throughout-Latency Tradeoff in LLM Inference with Sarathi-Serve
Amey Agrawal, Nitin Kedia, Ashish Panwar, Jayashree Mohan, Nipun Kwatra, Bhargav S. Gulavani, Alexey Tumanov, Ramachandran Ramjee
OSDI 2024 | May 2024
VIDUR: A Large-Scale Simulation Framework for LLM Inference
Amey Agrawal, Nitin Kedia, Jayashree Mohan, Ashish Panwar, Nipun Kwatra, Bhargav S. Gulavani, Ramachandran Ramjee, Alexey Tumanov
The Seventh Annual Conference on Machine Learning and Systems | May 2024
Just-In-Time Checkpointing: Low Cost Error Recovery from Deep Learning Training Failures
Tanmaey Gupta, Sanjeev Krishnan, Rituraj Kumar, Abhishek Vijeev, Bhargav Gulavani, Nipun Kwatra, Ramachandran Ramjee, Muthian Sivathanu
2024 European Conference on Computer Systems | April 2024
Keratoconus Classifier for Smartphone-based Corneal Topographer
Siddhartha Gairola, Pallavi Joshi, Anand Balasubramaniam, Kaushik Murali, Nipun Kwatra, Mohit Jain
IEEE International Engineering in Medicine and Biology Conference (EMBC) | July 2022
Varuna: Scalable, Low-cost Training of Massive Deep Learning Models
Sanjith Athlur, Nitika Saran, Muthian Sivathanu, Ramachandran Ramjee, Nipun Kwatra
EuroSys 2022 | April 2022
Best Paper Award
RespireNet: A Deep Neural Network for Accurately Detecting Abnormal Lung Sounds in Limited Data Setting
Siddhartha Gairola, Francis Tom, Nipun Kwatra, Mohit Jain
IEEE International Engineering in Medicine and Biology Conference (EMBC) | October 2021
Unsupervised Clustering using Pseudo-semi-supervised Learning
Divam Gupta, Ramachandran Ramjee, Nipun Kwatra, Muthian Sivathanu
Eighth International Conference on Learning Representations (ICLR) | April 2020
Balancing efficiency and fairness in heterogeneous GPU clusters for deep learning
Shubham Chaudhary, Ramachandran Ramjee, Muthian Sivathanu, Nipun Kwatra, Srinidhi Viswanatha
Fifteenth European Conference on Computer Systems (EuroSys'20) | April 2020
Gandiva: Introspective Cluster Scheduling for Deep Learning
Wencong Xiao, Romil Bhardwaj, Ramachandran Ramjee, Muthian Sivathanu, Nipun Kwatra, Zhenhua Han, Pratyush Patel, Xuan Peng, Hanyu Zhao, Quanlu Zhang, Fan Yang, Lidong Zhou
13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2018) | October 2018
-
SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills
Amey Agrawal, Ashish Panwar, Jayashree Mohan, Nipun Kwatra, Bhargav S. Gulavani, Ramachandran Ramjee
September 2023
-
Towards Automating Retinoscopy for Refractive Error Diagnosis
Aditya Aggarwal, Siddhartha Gairola, Uddeshya Upadhyay, Akshay P Vasishta, Diwakar Rao, Aditya Goyal, Kaushik Murali, Nipun Kwatra, Mohit Jain
August 2022
Singularity: Planet-Scale, Preemptive and Elastic Scheduling of AI Workloads
Dharma Shukla, Muthian Sivathanu, Srinidhi Viswanatha, Bhargav Gulavani, Rimma Nehme, Amey Agrawal, Chen Chen, Nipun Kwatra, Ramachandran Ramjee, Pankaj Sharma, Atul Katiyar, Vipul Modi, Vaibhav Sharma, Abhishek Singh, Shreshth Singhal, Kaustubh Welankar, Lu Xun, Ravi Anupindi, Karthik Elangovan, Hasibur Rehman, Zhou Lin, Rahul Seetharaman, Cheng Xu, Eddie Ailijiang, Suresh Krishnappa, Mark Russinovich
February 2022
-
SmartKC: Smartphone-based Corneal Topographer for Keratoconus Detection
Siddhartha Gairola, Murtuza Bohra, Nadeem Shaheer, Navya Jayaprakash, Pallavi Joshi, Anand Balasubramaniam, Kaushik Murali, Nipun Kwatra, Mohit Jain
IMWUT | October 2021, Vol 5(4)