_1722956449156001fz1R.png "Anyscale Logo")
A principal challenge in advancing scientific computing on campuses today is the efficient computation of a mix of conventional High Performance Computing (HPC) workloads with fast-growing Machine Learning (ML) workloads. Over the past two decades efficient campus computation has been realized by deploying collections of big data processing frameworks (Spark), data analysis tools (Pandas), and visualizers (Grafana), often cleverly combined to address a specific workload need. Yet this approach of integrating disjoint systems can be complicated. It is also poorly suited for higher education settings, where training on many tools reduces research productivity, strains campus research computing expertise, and taxes already thinly stretched IT finances. In addition, new user demand is surging, as less sophisticated data science and ML users from outside STEM disciplines need to learn and perform computational research in their fields of study.
In a pilot program at Princeton University, we are demonstrating how distributed execution with Ray can help support mixed HPC workloads in diverse on- and off-campus settings. We are exploring Ray software across a range of different computing environments, from 1) a shared campus research computing cluster; 2) a federated multi-institutional cluster across the NSF-supported FABRIC computing and networking research infrastructure; and 3) a hybrid campus and commercial compute cloud using the Google Cloud Platform (GCP). Taken together, the combined thrusts will demonstrate the extent to which a single, unified computing framework can support widespread coverage of diverse campus computational science needs.
In a pilot program at Princeton University, we are demonstrating how distributed execution with Ray can help support mixed HPC workloads in diverse on- and off-campus settings. We are exploring Ray software across a range of different computing environments, from 1) a shared campus research computing cluster; 2) a federated multi-institutional cluster across the NSF-supported FABRIC computing and networking research infrastructure; and 3) a hybrid campus and commercial compute cloud using the Google Cloud Platform (GCP). Taken together, the combined thrusts will demonstrate the extent to which a single, unified computing framework can support widespread coverage of diverse campus computational science needs.
