Generative AI that is driven by LLMs becomes critical for companies to succeed in the modern AI era. As a leading tech company in mobility and delivery, Uber has extensive domain-specific knowledge and many LLM use cases for improving magical user experience and employee productivity.
To achieve this, we are extending Uber's ML platform (Michelangelo) to support end-to-end LLMOps experience. We have established a scalable and interactive development environment capable of utilizing hundreds of A100 GPUs on Ray with great flexibility. By integrating various open-source techniques for LLM training, evaluation and serving, we have significantly enhanced our capability to efficiently develop Uber's custom models based on state-of-the-art LLMs such as LLama2.
Bo Ling is a Staff Software Engineer on Uber’s AI Platform team. He works on NLP, Large language models and recommendation systems.
Chongxiao Cao is a senior software engineer at Uber, focusing on enhancing distributed deep learning training infrastructure and optimizing high-performance data loading within the Michelangelo platform. He also serves as a leading contributor to the Horovod distributed deep learning framework and is a co-maintainer of the Petastorm data loading library.
Come connect with the global community of thinkers and disruptors who are building and deploying the next generation of AI and ML applications.