PDE Benchmark ★
Definition
Public datasets and metric suites for uniformly evaluating Physics-AI models, typically covering multiple PDE families, multiple parameter regimes and standardized train/test splits. A prerequisite for any credible claim that a model is better.
What it is
A PDE benchmark turns "our paper says we are better" into "better on the same data, same splits and same metrics". Typical ingredients: several PDEs (Burgers, Navier-Stokes, Darcy, heat conduction, elasticity), several parameter regimes per family, fixed train/validation/test splits including out-of-distribution splits, and unified error metrics (relative L2 and friends).
Key points
- Separate in-distribution from out-of-distribution: benchmarks such as PDEBench provide parameter-extrapolation tests; in-distribution numbers alone say nothing about generalization.
- Align metric definitions: whether relative L2 divides by the solution norm, whether it is averaged over time steps, whether fields are normalized — these choices change cross-paper comparisons dramatically.
- Baselines must be strong: compare against a naive "classical solver plus interpolation" baseline too; some published models have been shown to lose to it.
- Report compute cost (training time, parameter count, inference latency) alongside accuracy.
Common pitfalls / boundaries
- Good benchmark scores do not imply engineering readiness: benchmark cases usually have simple geometry and idealized parameters.
- Evaluating only on self-generated data is the most common evaluation flaw; always include third-party public datasets.