AI/ML Platform & Applications

Hyperparameter Tuning with Ray[Tune] for Next-Gen Training Platform at LinkedIn

September 18, 2:30 PM - 3:00 PM

LinkedIn has seen a surge in the use of machine learning over the past few years, driven by more advanced and sophisticated models such as LLM (Large Language Model), LPM (Large Personalized Model), and GNN (Graph Neural Network). These technologies are enabling LinkedIn members to identify new opportunities, receive personalized recommendations, and connect with other professionals in a more effective and efficient way. As a result, it has become necessary to design a training platform that can support these innovations, providing a flexible, user-friendly, and scalable training experience for our AI engineers.

In this talk, we will present LinkedIn's cloud-native deep learning training platform on Kubernetes; and how the training platform team leverages Ray(Tune) to enable hyper-parameter tuning in our training pipeline. We treat hyper-parameter tuning as a first class citizen in our training platform as we envision an AutoML end goal that will democratize AI for all engineers.

About Wei-Yu

Weiyu Yen is a software engineer at LinkedIn's Machine Learning Training Platform team. His team helps provide a seamless training experience for all AI modelers at LinkedIn.

Wei-Yu Yen

Machine Learning Infra. Eng., Linkedin
Photo of Ray Summit pillows
Ray Summit 23 logo

Ready to Register?

Come connect with the global community of thinkers and disruptors who are building and deploying the next generation of AI and ML applications.

Photo of Ray pillows and Raydiate sign
Photo of Raydiate sign

Join the Conversation

Ready to get involved in the Ray community before the conference? Ask a question in the forums. Open a pull request. Or share why you’re excited with the hashtag #RaySummit on Twitter.