Oxford University Press's
Academic Insights for the Thinking World

Collecting and evaluating data on social programs

On 31 December 2014, Ron Haskins of the Brookings Institution wrote a compelling op-ed piece in the New York Times entitled, “Social Programs That Work.” Haskins shared the need for our nation to support evidence-based social programs and abandon those that show small or un-enduring effects – a wise idea.

But one of the greatest challenges to identifying successful programs resides with existing programs’ capacity to collect and evaluate relevant data. Many smaller programs or those that aren’t optimally funded (and who really is “optimally funded” these days?) don’t have adequate systems in place to gather or analyze the information needed to determine the degree to which their programs meet stated goals. The result is that many programs that may be highly effective don’t have the opportunity to demonstrate that effectiveness. To add to this situation, the popular statistical software that is necessary to evaluate program data is often a budget-buster, but not having this critical tool can make or break a program’s ability to determine efficacy.

Now, imagine that you are the director of a social service agency. The economy is bad and the demands you face are great. Are you more likely to purchase a one-year license for statistical software or add more services? Not surprisingly, evaluations of social programs are often pushed by the wayside in favor of staffing programs and providing services.

One way around this dilemma is to train program administrators and evaluators to use R, a free open-source statistical programming language that can be used in place of expensive brand-named statistical software, such as SPSS.

R is an offshoot of a product developed at Bell Laboratories, and is currently maintained by collaborators, mostly academics, from all over the world. Virtually anything you would want to know or learn about it is freely available. Not only can you download R and its accompanying packages from www.r-project.org, but you can also get started there with downloadable manuals, a refereed journal, and a list of recommended books. A recent search on YouTube with the search criteria of “R + statistics” yielded 381,000 results. In the spirit of things being freely available, top-notch universities including Harvard, UC Berkley, and Johns Hopkins are offering classes on R via massive open online courses such as Coursera and edX. R is growing in popularity in a number of fields, including social services.

In contrast, SPSS (owned by IBM) is probably the most popular proprietary statistical software available and is far from cheap. The base package for an individual license is $2,530 for the first year and $5,760 per year thereafter — with no bells or whistles. The high-end version begins at $7,590 for the first year and is then $17,300 thereafter. If you want that version with multiple users on a network, that will cost you $19,000 for the first year and $43,100 each year thereafter. That is, unless the prices go up (because you know they’re not going down).

R is one tool that is accessible to everyone with a computer and an Internet connection. Programs whose staff have learned to use this are one step closer to being able to affordably evaluate the work they do. This is important to all of us who are invested in identifying effective social programs.

As professors, we and our colleagues have picked up on this, and an increasing number of social work programs are teaching students to use R in the hopes that, as students graduate and go into the field, they can build research capacity in the agencies in which they work. We recognize that this will take some time, but we believe it is well worth the effort.

Headline image credit: Office worker. CC0 via Pixabay.

Recent Comments

  1. Arthur Z

    What a terrific solution for developing better social programs — especially at a time when people want to make sure their tax dollars on public programs are well spent and effective.

Comments are closed.