Casting a safety net: a reliable machine learning approach for analyzing coalescing black holes

Tübingen, Potsdam – When two black holes merge, they emit gravitational waves that race through space and time at the speed of light. When these reach Earth, large detectors in the United States (LIGO), Italy (Virgo) and Japan (KAGRA) can detect the signals. By comparing against theoretical predictions, scientists can then determine the black holes' properties: masses, spins, orientation, position in the sky and distance from Earth.

A team of researchers from the Empirical Inference Department at the Max Planck Institute for Intelligent Systems (MPI-IS) in Tübingen and the Department of Astrophysical and Cosmological Relativity at the Max Planck Institute for Gravitational Physics (Albert Einstein Institute/AEI) in Potsdam has now developed a self-checking deep learning system that very accurately extracts information from gravitational-wave data. In the process, the system checks its own predictions about the parameters of merging black holes – a deep neural network with a safety net. A set of 42 detected gravitational waves from merging black holes were successfully analyzed by the algorithm: When cross-checked against computationally expensive standard algorithms, the results were indistinguishable. The study was published on April 26, 2023 in the journal Physical Review Letters.

A machine learning method with robustness guarantees: By comparing to simulated waveforms, the DINGO neural network verifies its own results. DINGO first calculates the properties of the black holes from the measured gravitational-wave signal. Based on these calculated parameters, a gravitational wave is modeled, and then compared to the originally observed signal. © M. Dax (Max Planck Institute for Intelligent Systems)

DINGO: a deep neural network for gravitational-wave analysis

The researchers have developed a deep neural network called DINGO (Deep INference for Gravitational-wave Observations) to analyze the data. DINGO has been trained to extract – or infer – the gravitational-wave source parameters from the detector data. There was a press release on this in December 2021. The network learned to interpret real (observed) gravitational-wave data after training with many millions of simulated signals in different configurations.

Trust, but verify

However, at first glance, it is not possible to tell whether the deep neural network is reading the information correctly. Indeed, one disadvantage of common deep learning systems is that their results sound plausible even when they are wrong. That's why the researchers at MPI-IS and AEI have added a control feature to the algorithm. Maximilian Dax, doctoral student in the Department of Empirical Inference at MPI-IS and first author of the publication explains: “We have developed a network with a safety net. First, the algorithm calculates the properties of the black holes from the measured gravitational-wave signal. Based on these calculated parameters, a gravitational wave is modeled, and then compared to the originally observed signal. The deep neural network can thus cross-check its own results and correct them in case of doubt.”

The algorithm controls itself, making it much more reliable than previous machine learning methods. But not only that. “We were surprised to discover that the algorithm is often able to identify anomalous events, namely real data inconsistent with our theoretical models. This information can be used to quickly ‘flag’ data for additional investigation,” says Stephen Green, co-lead author, and former Senior Scientist at the AEI (now at the University of Nottingham).

“We can guarantee the accuracy of our machine learning method – which almost never happens in the field of deep learning. It therefore becomes compelling for the scientific community to use the algorithm to analyze gravitational-wave data,” says Alessandra Buonanno, author and director of the Department Astrophysical and Cosmological Relativity at the AEI. Scientists from around the world are studying gravitational waves in large collaborations, such as the LIGO Scientific Collaboration (LSC), in which more than 1,500 researchers are organized.

Bernhard Schölkopf, who is a Director at MPI-IS, adds: “Today, DINGO analyzes gravitational-wave data – but such a self-controlling and self-correcting method is also interesting for other scientific applications where it is crucial to be able to corroborate the correctness of ‘black-box’ neural network methods.”

Membership in the LSC

Maximilian Dax and Jonas Wildberger, the two Ph.D. students at MPI-IS, are also members of the LIGO Scientific Collaboration. Through this membership, they can quickly access the gravitational-wave detector data, and collaborate with the relevant working groups. Their goal is to develop DINGO into a standard method for the analysis of gravitational-wave data.