Smoothing survey data based on different questions on the same topic
Subject matter
- Problems with comparability of existing survey data
- Survey questions on the same topic (e.g. happiness or life satisfaction) often differ in response format (e.g. a 3-point or a 5-point scale). This complicates comparison across surveys and over time; a large body of incomparable data has gathered dust in the archives of survey agencies.
- Survey studies often have different language versions, e.g. the World Values Survey uses questionnaires in 50 different languages. This language variation also complicates comparison: will the response option 4 ‘pretty happy’ on a 5-point scale for English speaking respondents be the same as for Chinese respondents ticking the same option 4 labelled as: 相当快乐 in their questionnaire?
- Solutions
At the Erasmus University Rotterdam, we have developed two new techniques for scale homogenization. We did this in the context of comparative happiness research, but the techniques are also applicable for other fields of research, such as survey studies on health or political attitudes. The techniques are:- The Scale Interval Method (SIM): Here native speakers rate the intensity of verbal response options on a numerical scale, e.g. in the context of a particular question the verbal response option ‘very happy’ equals 8.4 on numerical scale 0-10. Ratings are made by native-speakers; mostly students. Some 200 judges are required to obtain a reliable estimate. See http://worlddatabaseofhappiness.eur.nl/scalestudy/scale_fp.htm
- The Reference Distribution Method (RDM): Here the continuous distribution, estimated on basis of the responses to an appointed survey item, preferably with a 10- or 11-point numerical scale, serves as a reference to estimate the population mean based on the responses to another question on the same topic using a different response scale. This method requires that both questions have been used in the same population in the same year, e.g. that two surveys in the USA in 2015 have used a question on life satisfaction, one of which was rated on a 5-point scale with verbal response options, while the other was rated on a 0-10 point numerical scale.
E-course
Teachers
Topics
- Who we are
- Introduction to the problem
- Earlier solutions to the problem
- Scale Interval Method
- Reference Distribution Method
- Demonstration of an application
- Demonstration of use in World Database of Happiness
- Other applications of the methods
Materials
a) Book: Diversity in Survey Questions on the same topic: Techniques for improving comparability
b) Lectures
- Lecture 1: Overview. Free available at https://www.youtube.com/watch?v=_oM4pQY9d5A
- Lecture 2: Scale Interval method
- Lecture 3: Reference Distribution method
c) Programs:
- Program Processing Assessments Scale Interval Study
- Best Fit Beta Distribution for a Single Measurement
- Program Reference Distribution Method for Time Series
d) World Database of Happiness: https://worlddatabaseofhappiness.eur.nl
Costs
for copy of lectures 2 + 3 + Book + software: € 300 to be paid on invoice
Information and ordering
Erasmus Happiness Economics Research Organization EHERO,
E-mail: ehero@ese.eur.nl
Tel: +31 10 4082059
OTHER SCALE HOMOGENIZATION SERVICES by EHERO
- In house seminar: for your team € 500,-
- Data smoothing at Erasmus University: you send us the data and we do the smoothing; you can be present in this process. € 150,- per hour.
