Lightning Talks

Decentralized Ray - Scaling Ray for Global Inference

September 18, 5:00 PM - 5:15 PM

This presentation explores's revolutionary approach to achieving high-throughput inference by deploying thousands of decentralized, globally-distributed GPU clusters powered by Ray Core and Ray Serve. Demonstrations will include a step-by-step guide on rapidly scaling "KubeRay" and Ray Core clusters to meet any computational demands.

Key Insights:

Decentralization for Lower Latency: Understand how has used decentralized architectures to bring machine learning models closer to end-users, thereby reducing inference time.

Global Distribution for Resiliency: Learn about the importance of globally-distributed GPU clusters in achieving fault tolerance and maximizing uptime.

High Throughput with Ray Serve: Discover how Ray Serve can be tuned for high-throughput inference pipelines, allowing for more requests per second without sacrificing quality.

Rapid Deployment via KubeRay: Witness a live demo that showcases the speed and ease with which KubeRay enables deployment of Ray clusters, scaling to any size almost instantaneously. The world's first decentralized Kubernetes clusters.

Adaptability and Flexibility: Gain insights into the adaptability of this architecture, suitable for various industry applications.

About Ahmad

Co-founder and CTO of ANTBIT.IO, With a passion for distributed computing and DePIN, he has revolutionized the industry by building the world's largest decentralized GPU cloud, empowering generative AI startups. Ahmad's groundbreaking achievements include winning international medals for his GPU textures compression algorithm speeding up rendering by 66 times, and building low-latency ML Quant Trading Algorithms.

About Matej

Co-founder and COO of ANTBIT.IO. Prior to ANTBIT.IO he was director of FCA regulated crypto asset management platform in UK and prior to that he was involved in the finance industry for over 20 years. Starting out as a broker and portfolio manager, he went on to become a fund manager, the CEO of his own fund management company and the CEO of the biggest private fund management company in Slovenia. With wide-ranging interests, he has also cofounded start-ups in the food industry (organic and vegan products), IT (sensors, IoT), and digital marketing (virtual reality, social media campaigns).

Ahmad Shadid

Co-Founder & CTO,

Matej Tomazin

Co-Founder & COO,
Photo of Ray Summit pillows
Ray Summit 23 logo

Ready to Register?

Come connect with the global community of thinkers and disruptors who are building and deploying the next generation of AI and ML applications.

Photo of Ray pillows and Raydiate sign
Photo of Raydiate sign

Join the Conversation

Ready to get involved in the Ray community before the conference? Ask a question in the forums. Open a pull request. Or share why you’re excited with the hashtag #RaySummit on Twitter.