Ray Use Cases

From Spark to Ray: An Exabyte-Scale Production Migration Case Study

September 18, 1:45 PM - 2:15 PM
View Slides

This case study revisits the key milestones that allowed Amazon to successfully transition core exabyte-scale data catalog management jobs from Spark to Ray. We'll review the challenges encountered, concessions made, and future vision to more deeply incorporate Ray into critical batch and streaming business intelligence pipelines at Amazon. Topics covered include techniques for developing large-scale serverless Ray job management infrastructure, build and deploy management, risk mitigation strategies leveraged to ease the migration to Ray, and operational excellence methods employed to ensure a smooth rollout to production.

About Patrick

Software engineer working on data management and optimization for big data technologies at Amazon.

Patrick Ames

Principal Engineer, Amazon
Photo of Ray Summit pillows
Ray Summit 23 logo

Ready to Register?

Come connect with the global community of thinkers and disruptors who are building and deploying the next generation of AI and ML applications.

Photo of Ray pillows and Raydiate sign
Photo of Raydiate sign

Join the Conversation

Ready to get involved in the Ray community before the conference? Ask a question in the forums. Open a pull request. Or share why you’re excited with the hashtag #RaySummit on Twitter.