Project Launch: Assessing and Modelling Unit Nonresponse Bias in Probability-Based Web Surveys
The Centre for Social Informatics (CSI) has launched a new project "Assessing and Modelling Unit Nonresponse Bias in Probability-Based Web Surveys" led by Gregor Čehovin, PhD.
Project Background
Web surveys have become the predominant survey mode, especially when combined with nonprobability sampling. However, this proposal addresses academic, government and other social science research that relies on probability surveys using traditional survey modes (mail, telephone, face-to-face), which are now transitioning to probability-based web surveys (PBWS). In the context of total survey error, web surveys are already known for their strong measurement properties, while noncoverage is a diminishing issue, but nonresponse remains a major problem for PBWS.
Recruitment into PBWS relies on traditional survey modes, as there are no email lists for general population surveys, which are the primary domain of this project. However, to save costs, recruitment into PBWS is mostly carried out by mail (or email where possible), which significantly reduces response rates, which are often even below 20%. This is very low compared to traditional surveys, where response rates of 70% or 80% were once the standards that have been gradually abandoned (with little justification). Such low response rates increase the danger of nonresponse bias, which means that the estimates from PBWS can differ notably from the true values of the parameters. This presents a serious threat to the quality of empirical social science research.
Previous evaluations of PBWS have focused on sample representativeness in terms of control variables (e.g., R-indicator) or measurement equivalence. No systematic question-level evaluations have examined PBWS-specific nonresponse biases that may occur for some topics (e.g., poverty) but not for others (e.g., elections). We lack a thorough insight into the moderators associated with this bias.
Project Objectives
The main objective is to comprehensively elaborate and model the PBWS-specific nonresponse bias. The overarching research question is: Under what circumstances can PBWS be used instead of traditional surveys, given the impact of nonresponse bias?
Sub-objectives:
- Analyse and model the moderators of PBWS-specific nonresponse bias, particularly response rate but also research topic, respondent characteristics and methodological features;
- Predict PBWS-specific nonresponse bias for any question;
- Evaluate PBWS-specific nonresponse bias in the contexts of accuracy and cost.
The results will provide:
- A comprehensive elaboration of nonresponse bias in PBWS;
- Decision-making guidance on the suitability of PBWS for different objectives in social science research;
- A feasibility study for a stand-alone tool to predict bias in PBWS, complementing the Survey Quality Predictor (SQP), which predicts measurement error;
- Contributions to the transition of ESS 2025 to PBWS;
- Facilitation of the implementation of PBWS in Slovenian official and academic surveys.