DevConf.US 2022

Storage system failure is among the main culprits behind data center failure. It can lead to service degradation, resulting in poor customer experience, hiccups in business operations, loss of revenue, and even loss of data. So improving storage system reliability is often a top priority for organizations. But how exactly can this be done? We believe that analyzing disk health and predicting disk failure can significantly help in anticipating and mitigating storage system issues.

However, this has historically been a difficult task, especially in open source. Most disk health datasets are sourced from just one or a handful of data centers. Of these datasets, only a few are made publicly available. And in the ones that are publicly available, the data is representative of very few types of workloads.

But not anymore! We believe these pain points can be addressed with the Ceph Device Telemetry Dataset. This open source dataset contains SMART metrics collected from disks running in thousands of Ceph clusters, spanning >20 vendors and >2000 models. Since Ceph is used by several types of organizations and individuals, this dataset can capture usage patterns for a wide variety of real world workloads. Moreover, it is a continuously growing dataset not just in terms of number of datapoints, but also in terms of the metrics (features) collected. This makes it a go-to dataset for the disk failure analysis problem that is of great interest in both academia and industry.

In this talk, we will introduce the Ceph telemetry dataset and show how we visualize cluster and device trends on dashboards. Then, we will show how to set up data science workflows to extract insights on disk behavior, and implement machine learning approaches to anticipate disk failure. Finally, we will describe how you can contribute to this effort as a data scientist, a domain expert, or a Ceph user.