CSI members attended the AAPOR 74th Annual Conference
Asist. Miha Matjašič, Gregor Čehovin, PhD and Jernej Berzelak, PhD from the Centre for Social Informatics (CSI) attended the American Association for Public Opinion Research’s (AAPOR) 74th Annual Conference. The conference took place in Toronto, Canada, from 16th to 19th May 2019. This is one of the main global symposia in the field of survey methodology and survey public opinion. At the conference there were many articles and discussions on the topic of web surveys and the challenges on how to improve data quality, i.e., how survey research is conducted and disseminated.
At the conference asist. Miha Matjašič, presented the paper "Web surveys: using response time to identify low response quality speeders?" which was prepared in collaboration with Vasja Vehovar, PhD. The purpose of the paper is to address the phenomena of identifying low quality speeders in web surveys due to the fact that many reasons exist to eliminate the extreme speeders from the survey, particularly when they exhibit low response quality. However, to unambiguously identify low response quality speeders is a difficult task which is often ignored when various approaches were developed for eliminating the speeders. Namely, very often these approaches rely on some arbitrary technical criteria, for which they only pre-assumed to have some relation to response quality. However, this relation is often ambiguous or nonexistent. Therefore the presented paper addressed this issue starting with response quality perspective and not with response time perspective. In the empirical study (n=1440) we thus studied how the units with low response quality relate to corresponding response times and speeding. For this purpose, we defined eight response quality indicators and observed the overall response quality consequences in case we removed different shares of the fastest speeding units. The results showed that eliminating 0.5% of the fastest units was the optimal share, where we truly removed the units with the lowest (and often also unacceptable) response quality. On the other hand, this share also prevented that we did not removed the fast units (speeders) with still acceptable response quality, which is a typical mistake with other approaches. Of course, this specific share (0.5%) might be survey specific. The results show that in order to identify the optimal share of speeders – which removal would increase the overall response quality – in any specific survey, researchers should combine response times with response quality indicators (e.g., item nonresponse, straight-lining).
> Conference rogram and abstracts