Stability, cross validity, and external validity of risk preference measures: experimental evidence from a UK representative sample
Speaker(s): Sara Ribeirinho Machado
Date: Thursday, 2 June, 2016
Time: 10.45-11.45
Venue: H12-30
Contact person(s): Teresa Bago d'Uva
Abstract
We report evidence from an “artefactual” field experiment that directly incorporates three different experimental measures for risk preferences within the Innovation Panel (IP) of the UK Household Longitudinal Survey (UKHLS, also known as Understanding Society), the world-largest multi-scope panel survey. We randomly allocated to an experimental risk preferences module a representative sub-sample of 661 respondents to the IP Wave 6 (IP6). The selected subjects responded to three tests: the incentive-compatible (IC) binary-lotteries multiple price list test proposed by Holt and Laury (2002, 2005) (HL); the IC multiple-lotteries method originally proposed by Binswanger (1980, 1981) and further developed by Eckel and Grossman (2005) (EG); and the German SOEP survey test proposed by Dohmen et al. (2011) to measure self-reported willingness to take risk in general, and in financial and health matters (SOEP). One year after, at IP Wave 7 (IP7), we repeated the same experimental measures for risk preferences for the same sub-sample of respondents, being able to link responses across the two waves for 413 subjects.
This design allows us to systematically look, for a representative panel of the UK population, at the validity and stability of the three measures of risk preferences along three dimensions. First, we look at the “temporal stability” of risk preferences measures by comparing the responses of the 413 subjects across the two subsequent waves of data collection, at a distance of one year from an interview to the other. Second, we look at the “cross-validity” of the risk preferences measures by testing how individual responses at one point in time correlate across the HL and the EG tasks, and between these IC tests and the various self-reported measures. We also structurally estimate risk preferences assuming a Constant Relative Risk Aversion (CRRA) utility function, and calculating individual-specific levels of daily “background income” from linked survey data. Third, we look at the “external validity” of the three measures of risk preferences by linking the responses to a broad range of survey data in the UKHLS and by focusing on risky health behaviors.
We have three main findings. First, concerning “temporal stability”, there are significant but low correlations between the responses to the EG, HL, and SOEP tasks across IP6 and IP7. However, when presented exactly the same EG task in IP7, less than one third of the respondents chose the same option they preferred in IP6, with the proportion of stable responses being higher only for the “safe” option. These figures are even lower for the HL and the SOEP tasks. Second, concerning “cross-validity”, we illustrate that it is important to account for the fact that respondents in a representative sample can integrate the monetary prizes from the EG and HL experimental tests within individual- specific levels of background income. Because of such an integration, the ranges of the CRRA values implied by the EG and HL tasks do not perfectly overlap to each other. Even accounting for such an integration, we find that within- subjects responses did not systematically map into each other between the different risk preferences measures. At least two thirds of the subjects made inconsistent choices in cross-validity terms, in the sense that the CRRA values implied by their responses to the HL task were incompatible with the range of CRRA values implied by their own responses in the EG task, once the individual-specific levels of background income are accounted for. Structural estimations return an estimated CRRA coefficient that is significantly and substantially higher in the EG task than in the HL task. Regression analysis also show that, while the responses to the EG task are positively and highly significantly associated to the SOEP measure in financial matters, there is little association between the SOEP measures and the HL task.
Finally, considering a broad range of health-related variables, we find mixed evidence concerning the “external validity” of the three risk preferences measures, especially for the EG and the HL tasks.