Articles

Numeric Estimation and Response Options: An Examination of the Accuracy of Numeric and Vague Quantifier Responses

---

Abstract

Many survey questions ask respondents to provide responses that contain quantitative information, often using either numeric open-ended responses or vague quantifier scales. Generally, survey researchers have argued against the use of vague quantifier scales. However, no study has compared accuracy between vague quantifiers and numeric open-ended responses. This study is the first to do so, using a unique data set created through an experiment. 124 participants studied word lists of paired words, where the experiment employed a 2 (context) x 2 (response form) x 6 (actual frequency) factorial design, with the context and form factors manipulated between subjects, and the frequency factor manipulated within subjects. The two conditions for the context factor are same-context and different-context conditions where the context word either was the same or different for each presentation of the target word. The other between subject factor was response form, where participants responded to a recall test using either vague quantifiers or numeric open-ended responses. Translations of vague quantifiers were obtained and used in accuracy tests. Finally, a numeracy test was administered to collect information about respondent numeracy. Different accuracy measures are estimated and analyzed. Results show context memory did not have a significant effect. Numeracy has an effect, but the direction depends on form and context. Actual frequency had a significant effect on accuracy, but did not interact with other variables. Importantly, results suggest vague quantifiers tend to improve accuracy more often relative to numeric open-ended response.

Keywords: response options, numeric cognition, vague quantifiers, recall