Ray Summit 2024

The number of operational satellites in Earth’s orbit has grown exponentially, from fewer than 1,400 in 2015 to nearly 10,000 today, and more than 100K satellites are anticipated by 2030. Scaling decision-making processes and realistically modeling interactions among spacecraft, each with their own mission constraints and capabilities, is a central challenge to managing the space domain. To evaluate potential courses of action, we are training reinforcement learning (RL) agents to achieve operationally relevant spaceflight mission objectives in a high-fidelity synthetic space environment. We will discuss how we adapted our high-fidelity astrodynamics simulation engine, which was initially designed for local execution, into a near-infinitely scalable Ray-based training pipeline in this presentation. We developed a modular, multi-layered, multi-agent, multi-mission RL Gym that allows agents to train in a variety of simulated conditions. We will demonstrate how the approach enables general-purpose pre-training using low-fidelity dynamics to learn approximate high-level behaviors, followed by fine-tuning in higher-fidelity environments for specific applications. We illustrate how RL agents trained in this environment are able to perform sophisticated inter-satellite operations.