|
Article Excerpt
How do nonprofit organizations use data and research? What challenges do they face in conducting research and managing data? In spring of 2004, 80 nonprofit organizations in Toledo, Ohio returned a survey on their research and data needs and practices. The survey found that nonprofits collect data on a wide variety of topics, but do not use much of the data that they collect, and do not collect much data that could be useful for other groups, particularly neighborhood organizations. The average nonprofit in the survey has five employees and four volunteers who, together, spend 56 hours per week collecting, managing, and reporting on data. Nearly half of the organizations have no staff or volunteers with formal research training. The others have only one or two people with formal research training. More than half indicated a need for training on how to conduct evaluations, how to use data management software, how to conduct research, and how to find funding.
Keywords: nonprofits, research methods, data management
Introduction
Of all the capacity issues facing nonprofit organizations in the United States, perhaps none has been so neglected as their research data needs. There are numerous training and technical assistance resources for various programming areas, organizational development, and information technology. But, outside of evaluation research, the crucial area of collecting, managing, analyzing and using research data is neglected by the trainers and the researchers working with nonprofits. And that is a serious neglect. Grant proposals, evaluations, fiscal monitoring, resource allocation, and overall project planning are all dependent on tight research and solid information. The types of research data needed for those activities can range from census data to client case data to all varieties of specialty data for different nonprofits, making research data practice a highly complex and time-consuming part of nonprofit work.
What do I mean by "research data practice"? Basically, I am concerned with the systematic collection of information to inform nonprofit program development and evaluation. This includes typically-recognized research practices such as surveys. It also includes systematic collection of client or membership data that could be, though it is often not, analyzed for patterns or categories. Research data practice does not have to be informed by a theoretical or disciplinary-based question and, instead, may be informed by a practical need such as "what are the best practices in the area of youth programming" (Stoecker, 2005). This research focuses on what research data nonprofit organizations collect, how they use that data, and what challenges they face in collecting and using research data. It is important to understand that the concept of research data includes most of the information that nonprofits collect, including client-level data that may not be viewed as research data by the organization. Such information, however, can often be very informative in helping nonprofits understand trends in who they are serving and in supplying the sampling frame from which they can conduct evaluation research. That such data is often not seen as research data is part of the problem.
The lack of focus on nonprofits' research data practice needs has real consequences. This research project began as an outgrowth of my experience conducting participatory evaluation research with a local neighborhood organization. As we attempted to collect outcome data for the organization, we realized how little data there was, and how difficult it was to fill the data gaps. Consequently, a group of us concerned about this issue set out to determine the state of the art of nonprofit information management.
The Lack of Data on Nonprofit Research Needs and Practices
We know little about the research data needs, practices, and capacities of small and medium size nonprofit organizations. In fact, in the broad area of nonprofit research, research data practice is almost completely uncharted. That doesn't mean we lack knowledge about research and data management methods that nonprofits could use. Indeed, there are voluminous literatures on everything from needs assessments and asset assessments to evaluation and a wide variety of other data practices in between that are common to nonprofit organizations. But we simply have no knowledge of the extent to which nonprofits effectively deploy those methods to collect, manage, and use research data and other information. There is only one published case study of a nonprofit's information management strategies (Houchin, 2002), and one other published report on an attempt to build nonprofit research and data capacity (Dattalo, 1998).
We do have some research suggesting that nonprofits' needs in the area of research data practices are pronounced. Perception gaps between nonprofit service recipients and providers, for example, not only exist, but are often unrecognized, particularly for neighborhood organizations (Kissane & Gingerich, 2004). And it is in neighborhood organizations where the research data deficit may be most pronounced because most data in the nonprofit realm is not coded by neighborhood. Nonprofits engaged in policy battles also need to learn about and engage in policy research methods to have any hope of impacting social policy (Fox, 2001; Appleton, 2003). And nonprofit management experts (Letts, Ryan, & Grossman, 1999; Bryson, 1995) emphasize the importance of research for effective nonprofit mission accomplishment.
It is in the arena of evaluation where research and data management issues have been most discussed (Mancini, Marek, Byrne, & Huebner, 2004). Without developing their own data and research capacity, nonprofits are at the mercy of outsiders who can impose performance measures in support of increasingly popular "pay for performance" schemes (Theuvsen, 2004). The antagonism generated by most funder-directed evaluation schemes leads nonprofits and funders to play a cat and mouse game with research data, with nonprofits often reduced to selectively sharing information, enlisting experts to massage information, or generating information that is only symbolic (Ebrahim, 2002).
There is continuous argument over how outsiders can measure nonprofit organization effectiveness (Sowa, Selden, & Sandfort, 2004; Wing, 2004). Social accounting is expanding in popularity as a way to measure the social and economic value of nonprofit and community organization activities (Mook, Richmond & Quarter, 2003). More common are attempts to measure the economic impact of nonprofit activities, particularly by community development organizations (Woller and Parsons, 2002). There is pressure, however, to shift from output evaluation to outcome and impact evaluation (McNamara, 1999; United Way of America, 2005), vastly expanding the thorny methodological issues facing nonprofits.
There is also an expanding debate over who should be doing evaluation. Wadsworth (1991) urges organizations to integrate evaluation research into their daily activities in order to become more self-directed. Others have promoted empowerment evaluation and participatory evaluation, which better protect organizations against admitting failure because the model actually helps organizations achieve greater success (Fetterman, 2002; Patton, 1997; Millett, 1996). But while there is plenty of urging going on, there is only one published case of an evaluation requested by a nonprofit organization to address a...
|
