Ray Summit 2024

Generative AI promises a spectacular increase in productivity for enterprises that can leverage it on their proprietary/IP data or for users who can use it on their private data and within their personal accounts. However, the fear of exposing such confidential data to external parties (such as the LLM provider and its platform, its partners or employees, hackers breaking in the LLM platform, or subpoenas) have prevented organizations from using generative AI to its potential. For example, Samsung and Google blocked the use of ChatGPT internally, and many companies are still not using generative AI for their core proprietary data. In this talk, I will describe a novel technology based on confidential computing that can keep proprietary/private data encrypted throughout the entire generative AI cycle, and thus protected from the external parties above. I will overview research from my lab at UC Berkeley, as well as a real-world artifact that is both easy to use and does not degrade the quality of predictions. For example, our technology enables companies to keep their prompts confidential while using LLMs.