The Mechanics of Delta Learning: Target Design for Generalizable Scientific Machine Learning

This paper investigates the fundamental mechanics of Δ-learning in scientific machine learning, where models are trained on residual errors relative to physical baselines. Through evaluation of molecular graph neural networks on total energy prediction targets, the research demonstrates that residual scale alone is an insufficient heuristic for predictabl learnability. The findings suggest that complex local descriptor baselines can yield small residual targets that are disproportionately rough, challenging the assumption that more accurate baselines inherently improve downstream performance in molecular systems.