Identifying anomalies in national population data

In summary

Professor David Blake and his fellow researchers developed methods for identifying and correcting historical anomalies in national population and mortality data.

The work has led to a change of practice at the UK Office for National Statistics (ONS), as well as at pension funds, life insurers, and actuarial consultancies.

Adoption of methods developed through the research has contributed to a reduction in prices charged by insurers for the transfer of pension liabilities, with the resulting benefit of large savings for UK pension funds.

What did we explore and how?

The 2011 census update of ONS population estimates for England & Wales raised the possibility that there were considerable flaws in the way population data was measured in both the UK and other countries that estimate their population size using periodic censuses.

The research "Phantoms Never Die: Living with Unreliable Population Data" found that the standard practice of reporting population size as the mid-year estimate was flawed because it assumes that births and deaths are spread evenly throughout the year. This turns out to be a poor assumption for years where births and/or deaths are concentrated in one part of the year.

Blake et al. developed methods for identifying such errors in the data. These included a set of graphical diagnostics that help pinpoint anomalies, a Bayesian model to quantify their size, and techniques to enable the detection of small systematic errors.

Benefits and influence of this research

The analysis of national mortality trends is critically dependent on the quality of population and mortality data. The methods developed through this research have been shown to significantly improve the researchers' ability to identify and quantify anomalies within data, with a consequent improvement in the data and subsequent analysis of.

The methodology developed in the course of this research has been adopted by organisations in the UK, France, and the United States.

The research prompted the Office for National Statistics (ONS) in the UK to acknowledge weaknesses in its previous practice, and to consider recommendations for the planning and execution of the 2021 Census.

The Continuous Mortality Investigation (CMI), by the Institute and Faculty of Actuaries (IFoA) carries out research into the mortality and morbidity experience of pension funds and life insurers. The research highlighted the potential impact of discrepancies to its work, and led to the CMI adapting its procedures to identify anomalies in its data using the methodologies of “Phantoms Never Die”.

The US-based Human Mortality Database (HMD), the UK Universities Superannuation Scheme (USS), and the French office of the leading actuarial consultancy firm Milliman have all adapted the research to correct and improve their data. The smoothing assumptions in the research featured in the 2018 USS Actuarial Valuation, contributing to a significant financial impact. Lower charges by insurers for the transfer of pension liabilities have saved UK pension funds up to £1bn.