Ray Summit 2024

The generative AI revolution has transformed the world of large-scale deep learning infrastructure. Modern machine learning platforms must be ready to support pre-training for massive foundation models, memory-intensive fine-tuning for LLMs and diffusion models, as well as low-latency deployments for multi-billion-parameter models. Navigating this emerging landscape requires new techniques and methodologies, leavened with a thorough understanding of the still-nascent GenAI tooling ecosystem. In this talk, we'll walk through how we've adapted and extended Netflix's production Ray platform to deal with these new challenges. We'll outline our experiences in deploying Ray for large-scale data processing & curation, LLM fine-tuning, foundation model training, and distributed inference. We'll also share our strategies for creating GenAI-ready ML infrastructure to support the needs of the modern deep learning landscape.