ANTIC: Adaptive Neural Temporal In-situ Compressor

Generative AI & LLMs

Published: arXiv: 2604.09543v1

Authors

Sandeep S. Cranganore Andrei Bodnar Gianluca Galleti Fabian Paischer Johannes Brandstetter

Abstract

The persistent storage requirements for high-resolution, spatiotemporally evolving fields governed by large-scale and high-dimensional partial differential equations (PDEs) have reached the petabyte-to-exabyte scale. Transient simulations modeling Navier-Stokes equations, magnetohydrodynamics, plasma physics, or binary black hole mergers generate data volumes that are prohibitive for modern high-performance computing (HPC) infrastructures. To address this bottleneck, we introduce ANTIC (Adaptive Neural Temporal in situ Compressor), an end-to-end in situ compression pipeline. ANTIC consists of an adaptive temporal selector tailored to high-dimensional physics that identifies and filters informative snapshots at simulation time, combined with a spatial neural compression module based on continual fine-tuning that learns residual updates between adjacent snapshots using neural fields. By operating in a single streaming pass, ANTIC enables a combined compression of temporal and spatial components and effectively alleviates the need for explicit on-disk storage of entire time-evolved trajectories. Experimental results demonstrate how storage reductions of several orders of magnitude relate to physics accuracy.

Paper Summary

Problem

The main problem addressed by this research paper is the exponential growth of data storage requirements for high-resolution, spatiotemporally evolving fields governed by large-scale and high-dimensional partial differential equations (PDEs). This growth poses a severe bottleneck for modern high-performance computing (HPC) infrastructures, which are struggling to keep up with the increasing demand for storage.

Key Innovation

The key innovation of this paper is the introduction of ANTIC (Adaptive Neural Temporal In-situ Compressor), an end-to-end in-situ compression pipeline that addresses both the temporal and spatial axes of data compression. ANTIC consists of two main components: a physics-aware temporal selector that identifies and filters informative snapshots at simulation time, and a spatial neural compression module that learns residual updates between adjacent snapshots using neural fields.

Practical Impact

The practical impact of this research is significant, as it provides a principled and computationally efficient tool for the in-situ storage of high-dimensional scientific simulation data. ANTIC can achieve compression ratios exceeding 400× for turbulent 2D Kolmogorov flows and 10,000× for 3D BSSN evolved binary black hole merger simulations, while maintaining spatial fidelity across both use cases. This means that researchers and scientists can now store and analyze large-scale simulation data without the need for expensive and time-consuming data storage solutions.

Analogy / Intuitive Explanation

To understand the core idea of ANTIC, imagine a video camera capturing a high-definition video of a complex event, such as a storm. The camera captures a large number of frames, each with a high level of detail. However, most of these frames are similar, with only a few key frames showing significant changes. ANTIC works by identifying these key frames and compressing the data in between them, effectively reducing the amount of data that needs to be stored. This approach is similar to how our brains process visual information, where we tend to focus on the most important details and filter out the rest.

Paper Information

Categories:

cs.LG

Published Date:

arXiv ID:

2604.09543v1

Quick Actions

Back to Home