Supercomputing in Deutschland

When the Dam Breaks

SIDE as a Matchmaker Facilitating CIRCE-JSC Collaboration for HPC-Based Flood Research in Duisburg

The loss of life and property brought on by unforeseen events or calamities, whether man-made or natural, has never been easy for humanity to prevent. It may be clear where the highest risk areas are, such as residential areas near a river bank, but knowing in advance what measures to take and how to strike the ideal balance between safety and missed opportunities is open to interpretation. This struggle is further complicated by the advent of climate change, which increases both the severity and unpredictability of extreme weather.

High-performance computing (HPC) has the potential to help humanity minimize these catastrophe related losses in life and property by simulating scenarios accurately enough to quantify the trade offs between safety and missed opportunities. As part of the ‘Computational Immediate Response Centre for Emergencies (CIRCE)’ project, the High-Performance Computing Center Stuttgart (HLRS) and the Duisburg Fire Service developed a simulation to predict flooding on the Rhine. Meanwhile, the Simulation and Data Laboratory Terrestrial Systems at the Jülich Supercomputing Centre (JSC) and HLRS’s Department of Numerical Methods and Libraries both aimed to use HPC-powered numerical simulations to analyze flood scenarios and identify high-risk evacuation areas.

Recognizing this shared objective, SIDE as a member of the EuroCC2 project brought these two research groups together. By fostering information exchange, SIDE enabled the development of a more effective solution for predicting the impact of dam break scenarios in Duisburg, Germany.

The simulations are performed in two phases. The first phase is called the flood test phase, in which the water flow is simulated under normal conditions to achieve a steady-state flow, i.e., when there is no temporal evolution of water depth or velocity. This first phase provides realistic initial conditions for the state of the river prior to the dam break simulation. Figure 1 shows some results of such a simulation. The second phase then introduces a breach in the dam, where a surge of water is simulated entering the downstream areas, as shown in Figure 2.

Nebeneinander angeordnete Ausgaben einer Hochwassersimulation. Links: ein Zeitreihendiagramm von „Mass flux of water“ gegenüber „Time“ mit drei farbigen Linien und einer Legende: Inlet (blau), South (orange) und Outlet (grün). Die orangefarbene South-Linie liegt am höchsten und nimmt im Verlauf der Simulation langsam ab, während die grüne Outlet-Linie deutlich niedriger liegt und nur kleine Schwankungen zeigt; die blaue Inlet-Linie liegt nahe an der grünen und ist schwer zu unterscheiden. Rechts: eine Hochwasserkarte mit einem dunkelblauen, geschwungenen Fluss und dem umgebenden Gelände, das von Grün über Gelb bis Rot eingefärbt ist. Die Karte ist mit „South“ oben links, „Outlet“ oben rechts und „Inlet“ unten beschriftet. Ein vertikaler Farbbalken rechts zeigt, dass verschiedene Farben unterschiedliche hochwasserbezogene Simulierungswerte darstellen, aber die Zahlenbeschriftungen der Skala sind in diesem Bild zu klein, um gelesen zu werden; daher kann die genaue Bedeutung und Richtung der Farbskala (zum Beispiel, ob Rot eine größere Tiefe/Fließgeschwindigkeit als Grün bedeutet) aus dieser Datei nicht bestätigt werden.
FIGURE 1: (Left) Net mass flux of water flowing at the boundaries of the computational domain (inlet, outlet, and south), which remains almost constant over time - steady state achieved. (Right) Visualization of the flood test phase in steady state conditions; black boxes point out the boundaries of the computational domain (south, outlet, and inlet, respectively) where the water flows.
Ein maßstabsgetreues Tischmodell, das die flussabwärtige Ausbreitung einer Überflutung in Wohngebieten nach einem Dammbruch veranschaulicht. Die leuchtend grüne Grundfläche stellt die Geländeoberfläche dar, und zahlreiche kleine weiße rechteckige Blöcke unterschiedlicher Größe und Höhe stehen für Wohnhäuser und andere Gebäude. Das dichte Raster aus Zwischenräumen zwischen den Blöcken bildet straßenähnliche Wege und verdeutlicht, wie das durch den Bruch freigesetzte Hochwasser flussabwärts strömen, sich durch die Quartiere ausbreiten und sich in tieferliegenden oder stärker eingeengten Bereichen sammeln kann, während es zwischen den Baukörpern hindurchfließt.
FIGURE 2: Visualization of downstream flood propagation in residential areas following a dam breach.

Figure 2 shows the visualization of downstream flood propagation following a dam breach. The dam breach point is marked with a black circle. The red-colored region indicates the river, and the green-colored region shows the water flowing downstream through residential areas. A small blue patch near the dam breach point shows some landscape where water has not yet covered the ground.

Another outcome of this SIDE matchmaking was the evaluation of two open-source codes, SERGHEI and OpenFOAM , for performing numerical simulations of dam break scenarios. Although OpenFOAM offers reasonable results, SERGHEI was chosen as the code to model the dam break use case because of its high degree of accuracy, optimized performance, and ability to run on Graphical Processing Units (GPUs), which can be significantly faster than the Central Processing Units (CPUs) that are typically used by OpenFOAM. Table 1 shows the difference in the simulation times for a flood test phase using CPUs and GPUs. It shows that the simulation time on GPUs for a simulation with SERGHEI is significantly lower, meaning that, in addition to energy savings from needing less time, less energy is needed because GPUs are themselves more energy efficient than CPUs for this type of simulation.

Resource used by SERGHEI    Runtime
CPU (AMD EPYC 7742 64-Core-Prozessor)    18995,9 (s)
GPU (NVIDIA A100-SXM4-40GB)    2344,27 (s)

Thanks to SIDE matchmaking, these German research groups not only strengthened their collaboration but also shared their expertise, solved their problem faster, and made better use of HPC and energy resources. Their collaboration is still ongoing, with the potential for future scientific and academic benefits as they learn more about making simulations faster and more energy efficient. Moreover, collaborations like this could eventually help better guide local authorities through their planning for similar unforeseen events and crises.

Weitere Beiträge