DevConf.US 2022

In today’s environment, enterprises across multiple industries face challenges that include:

* New and more complex calculations due to regulatory demands, product discovery, environmental modeling, etc.
* Explosive data growth and the need to move and analyze this data
* Limited on-premise compute capacity

To help solve these challenges, Marius Bogoevici and Aric Rosenbaum have worked with industry experts and market leaders to design a forward-thinking, open source HPC solution to help enterprises better address their current and future needs.

In this presentation, which is based on real-world engagements with customers in the financial services industry to help them improve their market and credit risk calculations, we will demonstrate an open source HPC solution that:

* Scales the compute platform by leveraging the power of the hybrid cloud to perform new and more complex calculations in less time at a lower TCO
* Automates the setup of a multi-cluster compute environment across multiple datacenters via Ansible
* Uses the replication capabilities of Ceph to ensure proximity of data to compute
* Automatically schedules the computations to the optimal cluster in a transparent manner
* Orchestrates pre/post-calculation steps with a framework such as Apache Airflow