The paper “High-dimensional Ensemble Kalman Filter with Localization, Inflation and Iterative Updates” has recently been accepted in Quarterly Journal of the Royal Meteorological Society (QJRMS).

Data assimilation is a procedure to produce high quality estimates of the state variables of a dynamic system based on a numerical model and observations of the system. The ensemble Kalman filter (EnKF) is a popular data assimilation scheme in atmospheric and oceanic science. The standard EnKF may diverge due to both the limited ensemble size and the model bias. Recent statistical research has provided a deeper understanding of the influence of high dimensions and has extensive application prosperity for data assimilation in the era of big data.

In this paper, we propose to replace the sample covariance in the EnKF with a statistically consistent high-dimensional tapering covariance matrix estimator to counter the estimation problem under high dimensions. Specifically, the localization length scale is selected by minimizing the standardized squared Frobenius norm loss on the estimation of the forecast error covariance.  Meanwhile, the inflation method and the iterative update structure are incorporated to account for the model bias. A high-dimensional Ensemble Kalman Filter (HD-EnKF) with localization, inflation and iterative updates is developed. A flowchart of the HD-EnKF data assimilation scheme is shown in Figure 1.

Figure 1 A flowchart of the proposed high-dimensional EnKF (HD-EnKF) assimilation scheme that conducts the localization, the inflation and the iterative updates

The proposed paradigm is tested on a Lorenz-96 model with spatially correlated observation systems. The results demonstrate that the proposed method could improve the assimilation performance under multiple settings, with the trial-averaged RMSEs being very close to an Oracle setting. Neither the inflation nor the localization alone could obtain the assimilation performance comparable to the HD-EnKF scheme, especially for large p and mis-specified models. The HD-EnKF could counter the high dimensionality and the model bias quite well.

Figure 2 The trial-averaged RMSEs and their 5% − 95% quantile bands over the last 1000 steps as a function of forcing term F’ under p = q = 40, 100, 200 and n = 20, 30, 40.

The first author of this paper is Sun Hao-Xuan (Ph.D candidate at the Institute of Advanced Interdisciplinary Studies, Peking University), and the other authors include Wang Shouxia (Postdoc at School of Mathematical Science, Peking University, now research assistant professor at School of Statistics and Management, Shanghai University of Finance and Economics), Zheng Xiaogu (senior scientist at Shanghai Zhangjiang Mathematics Institute and International Global Change Institute, New Zealand) and Chen Song Xi (corresponding author). The research was partially supported by National Natural Science Foundation of China grants 12292980, 12292983 and 92358303. This research was also partially supported by the National Key Scientific and Technological Infrastructure project “Earth System Science Numerical Simulator Facility” (EarthLab).