Large Language Models

Enabling End-to-End LLMOps on Michelangelo with Ray

September 19, 1:45 PM - 2:15 PM
View Slides

Generative AI that is driven by LLMs becomes critical for companies to succeed in the modern AI era. As a leading tech company in mobility and delivery, Uber has extensive domain-specific knowledge and many LLM use cases for improving magical user experience and employee productivity.

To achieve this, we are extending Uber's ML platform (Michelangelo) to support end-to-end LLMOps experience. We have established a scalable and interactive development environment capable of utilizing hundreds of A100 GPUs on Ray with great flexibility. By integrating various open-source techniques for LLM training, evaluation and serving, we have significantly enhanced our capability to efficiently develop Uber's custom models based on state-of-the-art LLMs such as LLama2.

About Bo

Bo Ling is a Staff Software Engineer on Uber’s AI Platform team. He works on NLP, Large language models and recommendation systems.

About Chongxiao

Chongxiao Cao is a senior software engineer at Uber, focusing on enhancing distributed deep learning training infrastructure and optimizing high-performance data loading within the Michelangelo platform. He also serves as a leading contributor to the Horovod distributed deep learning framework and is a co-maintainer of the Petastorm data loading library.

Bo Ling

Staff Software Engineer, Uber

Chongxiao Cao

Senior Software Engineer, Uber
Photo of Ray Summit pillows
Ray Summit 23 logo

Ready to Register?

Come connect with the global community of thinkers and disruptors who are building and deploying the next generation of AI and ML applications.

Photo of Ray pillows and Raydiate sign
Photo of Raydiate sign

Join the Conversation

Ready to get involved in the Ray community before the conference? Ask a question in the forums. Open a pull request. Or share why you’re excited with the hashtag #RaySummit on Twitter.