Members of the Centre for Social Informatics published an original scientific article in Statistika: Statistics and Economy Journal
Gregor Čehovin and Vasja Vehovar have published an original scientific article titled "Probability-Based Web Panels for Official Statistics: Basic Insights and Analysis of the Bias of Survey Estimates" in Statistika: Statistics and Economy Journal. Comparing the estimates of probability-based web panels with those from established probability-based surveys, they found that nearly one-third of the panel estimates deviated substantially from the benchmark values, indicating that such panels are selectively suitable for official statistics.
The study combined two components. First, a survey of national statistical institutes across the European Union, conducted in 2022 and 2023, examined current practices, plans, and perceptions regarding such panels. Of the 19 responding institutes, about one-quarter reported having implemented or initiated a panel, and the most frequently cited barriers to adoption were a lack of knowledge and expertise, whereas cost was mentioned less often. Second, the authors evaluated the quality of estimates from a Slovenian probability-based web panel (the 1KA Panel, N = 1,628) by replicating questions from 12 traditional probability-based surveys used in official statistics, which yielded 651 comparable estimates. Using the corresponding survey estimates as benchmarks, they computed the relative bias of each panel estimate. Across all estimates, the median relative bias was 16%, and 205 of 651 estimates (31%) exceeded a relative bias of 10%. The proportion of biased estimates varied markedly by topic, from 0% for consumer opinion to 69% for information and communications technology use, with behavioral and factual measures showing more bias than attitudinal ones. The authors conclude that probability-based web panels are suitable for some official statistics topics but introduce substantial bias for others.
The study was funded by research projects J5-3100 (Evaluating probability and nonprobability online panels), V5-2157 (Probability web panels in national statistics for persons and households), and J5-50159 (Assessing and Modelling Unit Nonresponse Bias in Probability-Based Web Surveys), and the research programme P5-0399 (Internet research).