|
Article Excerpt
We identify and present original analyses of four methodological issues related to using Survey of Consumer Finances data sets and illustrate these issues with recent articles published in this journal. The issues are recognizing that the respondent is not necessarily the household head, reporting race and ethnicity in conformity with Survey of Consumer Finances and federal standards, using the repeated-imputation inference method to combine the five implicates in each survey year's data set, and discussing the use of weighted or unweighted data in multivariate analysis. We found a considerable variation in how authors dealt with these issues, which could hinder replication or comparison of research results. Authors and reviewers should consider methodological issues related to the Survey of Consumer Finances more carefully. Editor's note: After this refereed paper was accepted for publication, Jeanne M. Hogarth was asked to solicit comments and assume the role of moderator for the discussion essays. As editor as these commentaries, her observations immediately follow the article plus those of Darryl E. Getter and Sandra J. Huston.
**********
The Survey of Consumer Finances (SCF), sponsored by the Federal Reserve Board, is a rich and convenient source of information about household finances. It has been conducted every three years since 1983, employing a complex sampling system that provides a sizable number of households and allows estimations of the U.S. population as a whole (Aizcorbe, Kennickell, and Moore 2003). The public data set is free, and the substantial online assistance includes working papers and well-documented codebooks. A user-developed Web site with tips for researchers is also available (Hanna 2006). However, despite the apparent ease of use, care is needed in using and reporting the data because of the sampling procedures, the method of imputing missing values, and the nature of some demographic variables. As with any research, those who use SCF data need to make appropriate decisions, justify procedures, precisely define variables, and provide sufficient detail so readers can understand the findings and other researchers can replicate the study.
It might seem that precise reporting is not necessary when a researcher works with a widely used national data set that provides detailed online documentation. A reader might assume that SCF researchers employ standard variables and commonly agreed upon methods for analysis, making it possible to compare research across studies. However, our initial overview of 28 articles that used the SCF, published in the Journal of Consumer Affairs, Financial Counseling and Planning, Financial Services Review, and the Journal of Family and Economic Issues, revealed a number of problems and inconsistencies in the use of variables and description of methods. Six of the articles we reviewed appeared in the Journal of Consumer Affairs during the years 2001-2004. In this article, we focus on those articles, analyzing their treatment of the following four issues that we identified as particularly problematic.
* Recognizing that the respondent is not necessarily the household head.
* Reporting race and ethnicity in conformity with SCF categories and federal standards.
* Using the data of all five implicates and the repeated-imputation inference (RII) method to combine the implicates.
* Discussing the use of weighted or unweighted data in multivariate analysis.
Table 1 lists the articles we used in our analysis and summarizes how each dealt with the four issues. We do not propose that any lack of precision related to these issues affected the authors' conclusions. The general problems we discuss, using these articles as examples, are not limited to these articles nor to this journal but occur in many articles using the SCF, including some we have authored. We analyzed these issues because precision and ability to replicate and compare results are fundamental to research, and paying attention to these issues can improve future research using the SCF.
ANALYSIS
The Household Head vs. the Survey Respondent
In the SCF, the respondent and the household head are not necessarily the same person, and either partner in a couple household could be the respondent. Because of the detailed nature of the financial data the SCF gathers, interviewers are trained to establish who is the most financially knowledgeable person in the household and to interview that person (Lindamood and Hanna 2005, 2006). Therefore, the respondent is the most financially knowledgeable person. The household head, however, is an arbitrary designation. In the SCF, the male always is designated as the head for mixed-sex couple households and the older partner always is designated as the head for same-sex couple households.
Correctly identifying whose demographic data is being used--the respondent's or the head's--is particularly important when analyzing variables such as race and the various attitude variables because the SCF collects race and attitude variables only for the respondent. It would be inappropriate, for example, to study the relationship of age and attitude using the head's age because attitudes are those of the respondent. The mistake might not be limited to a small number of households. In our weighted analysis of the 2001 SCF, about 60% of the households included couples, and about 87% of those were husband-wife couples. The wife was the respondent in 46% of the husband-wife couple households.
None of the six articles that appeared in the Journal of Consumer Affairs discussed the distinction between the respondent and the head (Table 1). Four articles referred to the race of the head or the race of the household, both of which are inaccurate under the SCF because the race of the head is known only if the head is the respondent. If researchers assume that a spouse/partner is of the same race as the respondent, that assumption should be specified and only then should the race of the respondent be used as a proxy for other members of the household.
The article by Hogarth, Anguelov, and Lee (2004) generally was the most careful in describing methods, but the tables and the discussion of the variables described "household age" and "education of the household" without identifying whether the "household" data represented the respondent, the head, or a composite the researchers created. The article also referred to the race of the household, which the SCF does not measure.
Getter (2003) did not discuss any methodological details but listed the variables of the model he tested. The listing represented education and employment as that of the respondent. However, the discussion and findings were presented in terms of the household head. He did not discuss whose race was measured, but his article replicated a study by Canner, Gabriel, and Woolley (1991), which incorrectly listed race as that of the head. Getter stated, "Demographic variables such as age, race, and whether or not the respondent has a college degree are also included in the analysis," making it possible that the demographic variables reflected the respondent.
Lyons (2003) did not discuss how demographic variables were measured or the difference between the respondent and the head. In her presentation of multivariate models, she referred to income of the head but did not mention whose age, gender, and ethnicity was included. The discussion of the results referred to the age of the head, and two of the tables listed age, education, and income variables as related to the head, but other tables did not specify whether the variables were of the head or respondent.
Lee and Kwon (2002) reported using the age of the respondent in their table describing independent variables but did not identify whose education or ethnicity was used. Ethnicity, by definition, had to be of the respondent. However, given the mean value reported for age, it is likely that they used the age of the head. Using the head's demographic data to study the attitudes expressed by the respondent is inappropriate without justification. If both partners in couple households are of similar age, this error may not be important. However, our analysis of the 1998 SCF (which Lee and Kwon used) shows that 10% of married couples had an age difference of 11 years or more, a difference that could have a biasing effect on analyses of age and attitude.
Chien and DeVaney (2001) correctly stated that attitudes were provided by the respondent but did not discuss the distinction between the head and the respondent. They listed age as being of the head but did not identify whose education was used in the analysis. They reported ethnicity as that of the head, which is not possible because race/ethnicity data are available only for the respondent.
Xiao et al. (2001) included only a couple of households in their analysis of family business owners, but characterized the sample as including only family heads. They reported that unlike previous research on business owners, they did not use gender as an independent variable because 99% of their sample was male, correctly noting that the SCF coded the male as the head in mixed-sex couple households and the older person as the head in same-sex couple households. However, they did not report that the head was not necessarily the respondent or the business owner and did not report the proportion of respondents who were female. Their discussion referred to the characteristics of the "family business owner"; yet, the analysis apparently was based on the age, race, and education of the head. Using the bead's characteristics is problematic because those characteristics were used to analyze the SCF risk tolerance variable, a variable available only for the respondent. Although that variable is supposed to reflect a couple's risk tolerance, it is likely that it reflects the risk tolerance of the respondent in married couple households (Hanna and Lindamood 2005).
Race/Ethnicity of the Respondent
Race is a controversial topic (American Anthropological Association 1997), and even the title chosen to describe a group has importance. The 2001 and 1998 SCFs did not directly use the term race or ethnicity on the card that listed the respondent's choices of race/ethnicity categories. Instead, the researcher asked the respondent "which of these categories do you feel best describe you?" when showing a card that directed: "Please list your strongest identification first." Even though seven identity categories were offered to respondents, the SCF reports only four in the public data...
|
