Lightning Talks

Ray Serve for IOT at Samsara

September 18, 5:15 PM - 5:30 PM
View Slides

Samsara's machine learning infrastructure team shares how they used Ray serve to scale their cloud IOT inference platform developing computer vision and sensor fusion detection models.

The team will share pragmatic best practices learned creating production serve clusters. Providing demos such as: templates to provision a production cluster setup turn-key, common utilities we've developed, and dashboards for observability and maintenance.

In the IOT space, configuration and customization is key, we'll touch on how Ray serve helped Samsara ML engineering dev-velocity by simplifying the composition of complex pipelines to target our specific customer's use-cases.

(( Key Talking Points ))

(Section) How is Samsara's Machine learning platform special and why did we choose ray serve?

  1. For teams already familiar with ray, what's the transition process like?

  2. Applications are generally event detection pipelines with configuration and conditionals.

  3. Data comes in many forms - computer vision and sensor fusion.

  4. Enabling real-time inferences for our application: ex. relevancy.

(Section) Learnings from Deployment

  1. Guidelines for cluster configuration - ex. autoscaling, logging.

  2. Metrics to monitor - dashboards, what to alert on.

  3. Composing pipelines as code.

  4. How we maintain our clusters.

(Section) Utility Sharing - Give back to the community

  1. Terraform for EKS cluster setup

  2. Template/example ray cluster cluster configuration

  3. Example complex serve inference pipeline

  4. Utils like model LRU cache to share endpoints

About Brian

Brian Westphal is a senior machine learning engineer working on computer vision and sensor fusion technologies for IOT applications at Samsara. Brian is a former startup founder with a master degree in computer science, artificial intelligence from Georgia tech.

Brian Westphal

Senior Machine Learning Engineer, Samsara
Photo of Ray Summit pillows
Ray Summit 23 logo

Ready to Register?

Come connect with the global community of thinkers and disruptors who are building and deploying the next generation of AI and ML applications.

Photo of Ray pillows and Raydiate sign
Photo of Raydiate sign

Join the Conversation

Ready to get involved in the Ray community before the conference? Ask a question in the forums. Open a pull request. Or share why you’re excited with the hashtag #RaySummit on Twitter.