Measuring and Improving Social Impacts by Marc J. Epstein

Author:Marc J. Epstein
Language: eng
Format: epub
Publisher: Berrett-Koehler Publishers
Published: 2014-06-14T16:00:00+00:00

Experiments

When you have strong logical reasons to believe that your interventions will lead to desired results and you want to validate this belief, you can conduct an experiment. Many programs evaluate their effectiveness by looking at whether participants are better off after the program than they were before it. On its face, this approach seems to be a valid way to evaluate impact, and it provides a simple and easy way to learn about performance.

But there are two important problems with the before-and-after, or pretest/posttest approach. The main problem is that you won’t know if the changes you find are a result of your program’s work. Most social and environmental changes are very complex and are influenced by a large number of factors. Without examining these factors, you can’t know what caused the change.

The best way to try to determine whether an intervention had the expected impact is through an experiment. In an experiment, some subjects receive or participate in an intervention while others do not. In other respects, the two groups of subjects are very similar. If you take this approach, you can see whether both groups changed in a similar manner over the time period when you look at the differences between pre- and post-test results. If one group changed more than another, you can attribute the difference to the intervention the group received.

Randomized control trials, or RCTs, have become the gold standard in providing evidence that a program works. RCTs have two major requirements. First, they use a control group that does not receive the intervention for comparison with the treatment group. This group serves as a “counterfactual”—it provides a picture of what would have happened without any intervention. Many factors can affect impacts, so it is difficult to tell whether impacts resulted from the intervention or something else, such as changes in economic, social, or natural conditions that affected beneficiaries. A control group subjected to the same factors lets you identify which changes resulted from the intervention.

Second, subjects must be randomly assigned to treatments. If subjects select which group they are in, results can be unreliable—for example, some may decide that they want to attend a drug treatment program while others decide not to participate. If more attendees than nonattendees stop using drugs during the study, it might be because the treatment program was effective, but it might be because those who decided to attend were more motivated to stop using drugs in the first place. By randomly assigning who gets the treatment and who does not, you can be more confident that changes are the result of the program and not some other variable.

While RCTs have advantages over other methods in that they can provide reliable evidence regarding an intervention’s impact, they are useful only in very limited circumstances. In order for the RCT to produce useful information, the treatment, by itself, must be realistically capable of resulting in significant and measurable changes in impact. For the best chance of this happening, everything else that might affect the outcomes must remain as stable as possible.

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.