Based on the goals you determine to be most important to evaluate, what evaluation questions do you want to answer? For program evaluation, consider:
- Planning and Implementation: How well was the program planned? Is it being implemented as intended? This is often done through ongoing monitoring of key indicators. Sample Questions:
- What are we doing? When? Where? How much?
- Are we on track with time and resources?
- Are we delivering the program as planned? If not, why has it varied?
- Are we reaching the intended population of people?
- Did the program evolve during the process? How?
- Impact: Did the program have the intended results? Data collected as part of your ongoing process evaluation may inform your impact evaluation. Sample Questions:
- What is different as a result of our actions? How are the people we’re trying to help different as a result of what we did?
- What did we accomplish? Did we achieve our outcomes? Why or why not?
- Were there any unintended effects of the program?
Advocacy/policy evaluation questions may differ from program evaluation questions since the outcomes are different, with one set of outcomes often contingent on others. (For example, in order to provide people with equal access to high quality care, you first have to build public and political will to make policy and systems changes.) Depending on which of the following six outcome categories you are evaluating, you might have different questions:
- Shift in social norms: How did beliefs, attitudes, public behavior change? How effective is our media strategy in reframing the policy issue?
- Strengthened organizational capacity: Does the advocacy organization have improved capacity to communicate and promote advocacy messages? Do community members have this capacity?
- Strengthened alliances: Are there an increased number of partners supporting an issue?
- Strengthened base of support: Is the advocacy effort increasing public will among its target audiences?
- Improved policies: Is there adequate funding and other resources for implementing a policy?
- Changes in impact: Were social and physical conditions improved?¹
Be prepared to revise these questions as the evaluation proceeds.
1. Organizational Research Services, Jane Reisman, Anne Gienapp, Sarah Stachowiak. A Guide to Measuring Advocacy and Policy. In: Identify Your Outcome Categories: Prepared for the Annie E. Casey Foundation, Baltimore, MD; 2007. p. 16-22.