Deep learning systems are consistently producing state-of-the-art results for tasks using unstructured data such as text, images, video, and audio, but unlocking this value in production requires taking best-in-class pretrained models (GPT, BERT, ViT, etc.) and tuning them to your domain-specific datasets and tasks. Ludwig is a low-code deep learning framework (developed and open sourced by Uber into the Linux Foundation) that integrates and scales natively with Ray to declaratively fine-tune state-of-the-art foundation models, customized to your domain-specific business data – including tabular metadata, or any other feature types.
In this talk, we present 3 ways to fine-tune powerful foundation models like LLMs on your data in less than 10 lines of YAML using Ludwig on Ray:
Modify the weights of a pretrained model to adapt them to a downstream task.
Keep the pretrained model weights fixed and train a stack of dense layers that sit downstream of the pretrained embeddings.
Use the pretrained model embeddings as inputs to a tree-based model (gradient-boosted machine).
We explore the tradeoffs between these approaches by comparing model quality against the training time / cost, and show you how Ludwig leverages the advanced capabilities of Ray AIR to provide out-of-the-box scale and optimized performance on any hardware.
Key takeaways from this talk include:
Project repository: https://github.com/ludwig-ai/ludwig
Travis Addair is co-founder and CTO of Predibase, a data-oriented low-code machine learning platform. Within the Linux Foundation, he serves as lead maintainer for the Horovod distributed deep learning framework and is a co-maintainer of the Ludwig declarative deep learning framework. In the past, he led Uber’s deep learning training team as part of the Michelangelo machine learning platform.
Come connect with the global community of thinkers and disruptors who are building and deploying the next generation of AI and ML applications.