Mode effect

Abstract: In a choice experiment setup considering a non-market good, this paper adds to the literature on survey mode effects by providing evidence that internet surveys can be a viable alternative to traditional mail surveys when gathering feedback from a sample of respondents. The case study concerns preferences for protecting different types of landscape from road encroachment when building new motorways in Denmark. Two samples of respondents are surveyed—one by internet and one by mail. The performances of the two samples are compared over six different criteria; response rates, protest responses, demographics, preferences and WTP, estimation precision, and, finally, certainty in choice. Differences are observed for some of these criteria, implying that analysts should be aware that choosing internet over mail could be accompanied by a survey mode effect. However, the observed differences do not translate into significant differences in the unconditional WTP estimates. In most applied economic valuation studies of non-market goods, the main objective is in fact estimation of WTP. Hence, in the present case, the identified survey mode effects do not severely invalidate the applicability and continued use of the internet as a suitable means of collecting data for choice experiment economic valuation of non-market goods.

Abstract: Surveys increasingly use mixed mode data collection (e.g., combining face-to-face and web) because this controls costs and helps to maintain good response rates. However, a combination of different survey modes in one study, be it cross-sectional or longitudinal, can lead to different kinds of measurement errors. For example, respondents in a face-to-face survey or a web survey may interpret the same question differently, and might give a different answer, just because of the way the question is presented. This effect of survey mode on the question-answer process is called measurement mode effect. This study develops methodological and statistical tools to identify the existence and size of mode effects in a mixed mode survey. In addition, it assesses the size and importance of mode effects in measurement instruments using a specific mixed mode panel survey (Netherlands Kinship Panel Study). Most measurement instruments in the NKPS are multi-item scales, therefore confirmatory factor analysis (CFA) will be used as the main analysis tool, using propensity score methods to correct for selection effects. The results show that the NKPS scales by and large have measurement equivalence, but in most cases only partial measurement equivalence. Controlling for respondent differences on demographic variables, and on scale scores from the previous uni-mode measurement occasion, tends to improve measurement equivalence, but not for all scales. The discussion ends with a review of the implications of our results for analyses employing these scales.