Samsara's machine learning infrastructure team shares how they used Ray serve to scale their cloud IOT inference platform developing computer vision and sensor fusion detection models.
The team will share pragmatic best practices learned creating production serve clusters. Providing demos such as: templates to provision a production cluster setup turn-key, common utilities we've developed, and dashboards for observability and maintenance.
In the IOT space, configuration and customization is key, we'll touch on how Ray serve helped Samsara ML engineering dev-velocity by simplifying the composition of complex pipelines to target our specific customer's use-cases.
(( Key Talking Points ))
(Section) How is Samsara's Machine learning platform special and why did we choose ray serve?
For teams already familiar with ray, what's the transition process like?
Applications are generally event detection pipelines with configuration and conditionals.
Data comes in many forms - computer vision and sensor fusion.
Enabling real-time inferences for our application: ex. relevancy.
(Section) Learnings from Deployment
Guidelines for cluster configuration - ex. autoscaling, logging.
Metrics to monitor - dashboards, what to alert on.
Composing pipelines as code.
How we maintain our clusters.
(Section) Utility Sharing - Give back to the community
Terraform for EKS cluster setup
Template/example ray cluster cluster configuration
Example complex serve inference pipeline
Utils like model LRU cache to share endpoints
Brian Westphal is a senior machine learning engineer working on computer vision and sensor fusion technologies for IOT applications at Samsara. Brian is a former startup founder with a master degree in computer science, artificial intelligence from Georgia tech.
Come connect with the global community of thinkers and disruptors who are building and deploying the next generation of AI and ML applications.