Reduction of Used Memory Ensemble Kalman Filtering (RumEnKF): A data assimilation scheme for memory intensive, high performance computing
Advances in Water Resources
Faculty of Civil Engineering and Geosciences, Department of Water Resources Management, Delft University of Technology, Delft, Netherlands; Council for Scientific and Industrial Research, Water Research Institute, Accra, Ghana
Reduction of Used Memory Ensemble Kalman Filtering (RumEnKF) is introduced as a variant on the Ensemble Kalman Filter (EnKF). RumEnKF differs from EnKF in that it does not store the entire ensemble, but rather only saves the first two moments of the ensemble distribution. In this way, the number of ensemble members that can be calculated is less dependent on available memory, and mainly on available computing power (CPU). RumEnKF is developed to make optimal use of current generation super computer architecture, where the number of available floating point operations (flops) increases more rapidly than the available memory and where inter-node communication can quickly become a bottleneck. RumEnKF reduces the used memory compared to the EnKF when the number of ensemble members is greater than half the number of state variables. In this paper, three simple models are used (auto-regressive, low dimensional Lorenz and high dimensional Lorenz) to show that RumEnKF performs similarly to the EnKF. Furthermore, it is also shown that increasing the ensemble size has a similar impact on the estimation error from the three algorithms. © 2015 Elsevier Ltd.
Digital arithmetic; Kalman filters; Data assimilation; Ensemble Kalman Filter; Ensemble Kalman filtering; Floating point operations; Global models; High performance computing; Inter-node communication; Memory problems; Computer architecture; algorithm; data assimilation; Kalman filter; numerical model