Comparison of missing data imputation methods using weather data

Hafiza Nida; Muhammad Kashif; Muhammad Imran Khan; Madiha Ghamkhar

doi:10.21162/PAKJAS/23.228

Author(s)

Hafiza Nida Department of Mathematics and Statistics, Faculty of Sciences, University of Agriculture,Faisalabad, Pakistan

Muhammad Kashif Department of Mathematics and Statistics, Faculty of Sciences, University of Agriculture,Faisalabad, Pakistan

Muhammad Imran Khan Department of Mathematics and Statistics, Faculty of Sciences, University of Agriculture,Faisalabad, Pakistan

Madiha Ghamkhar Department of Mathematics and Statistics, Faculty of Sciences, University of Agriculture,Faisalabad, Pakistan

Abstract

Researchers and data analysts commonly experience challenges while dealing with missing data for analyzing large data sets in their respective field of studies. It is necessary to handle missing data properly to obtain better and more reliable outcomes about any research. The objective of this research is to evaluate different imputation techniques for handling missing observations occurred in the weather data. For this purpose, weather data of the variables: daily rainfall, maximum temperature (Tmax) and minimum temperature (Tmin) of 23 stations of Pakistan have been taken from Pakistan Metrological department for the years 1981 to 2020. There are about 14610 total observations of each variable while each variable has different number of missing observations, called as size of missingness, at different stations. The techniques: mean imputation, k nearest neighbors (KNN) imputation, predictive mean matching (PMM) imputation and sample imputation have been considered for the estimation of missing observations found while analyzing data of each station. The minimal value of root mean square error (RMSE) is considered to decide about station-wise imputation technique because the size of missingness varied from station to station. The KNN technique is the most appropriate to estimate the missing observations of the rainfall variables for all the stations while mean imputation technique is recommended for Tmax and Tmin data; as compared to other imputation methods.

Publication Details

Page(s) 327-336

DOI 10.21162/PAKJAS/23.228

Published Journal: Pakistan Journal of Agricultural Sciences, Volume: 60, Issue: 2, Year: 2023

Request the full text from PASTIC

Keywords

temperature, missing data, rainfall, root mean square error., imputation methods