Prepare to Evaluate
The Take Action Cycle shows evaluation at the end of the cycle, but in reality evaluation should be incorporated throughout your process of improving your community’s health.
As you work to build a healthy community, your evaluation will take on different purposes.
- As you Work Together, you’ll want to continuously evaluate your partnership.
- As you Assess Needs & Resources, you’ll want to consider what measures are available to monitor progress over time.
- As you Choose Effective Policies & Programs and plan to Act on What’s Important, you may use evaluation tools to gain insight into how your community should address priority issues.
- As you Act on What’s Important, you may use evaluation tools to improve a policy or program. Once your selected solution has been implemented, you’ll want to evaluate its impact.
- As you Communicate about your initiative, you’ll want to share your evaluation results effectively.
This guide focuses on both policy advocacy and program evaluation. Because changing policy is complex, multi-step, and incremental, what you measure and the evaluation tools you use are different from program evaluation. We’ll provide guidance on where advocacy/policy evaluation diverges from program evaluation throughout this guide.
The time, money, and resources you spend on evaluation are an investment in your overall efforts. Evaluation can be used for learning about what’s working so you can replicate or expand your efforts or for accountability to demonstrate that your policy or program is having the intended impact. Evaluation is as much a mindset as an event or activity. Take time to talk about how each person in your team or partnership views evaluation and how they value the process of using reflection and dialogue to gain insight about information gathered.
There are a few things you’ll want in place as you embark on an evaluation process:
Understanding of your readiness
Are you ready to evaluate? The Evaluability Assessment helps determine your program’s readiness.
Resources to support an evaluation
Evaluation cost estimates range from 5 to 7 percent or 15 to 20 percent of your total budget.1,2 How much you need to invest in evaluation depends on a variety of factors, including:
- The scope of your policy or program
- The number of outcomes you want to assess
- Who conducts the evaluation
- Your available evaluation-related resources
In some cases, you may not have the financial resources for external evaluation. Consider reaching out to local universities to see if they have any courses or student groups that focus on evaluation. This may be an opportunity to partner for the good of both groups -- students get valuable experience and a client reference, and you get evaluation support.
A core team to guide and inform the evaluation
Evaluation is a team effort. Assemble a core evaluation team that knows the community, will ask good questions, is thoughtful and reflective, and is committed to developing and guiding the process with as much neutrality as possible.3 Community members should be engaged in all phases of your evaluation activities. Those who are affected by poor health outcomes in the community have the direct experience to tell you whether or not your policy or program efforts are creating positive change.
Clarity about who will lead the evaluation
You will need to determine who will do the evaluation (e.g., your core team, staff, or an outside evaluator).
If you’re embarking on an innovative strategy and the path ahead is unclear, you may want to consider engaging someone experienced in developmental evaluation. The developmental evaluator is part of the team that is designing and testing the innovation. Their primary role is to bring evaluative thinking into the process.
Shared understanding of how and when the evaluation will be used
When your team is ready to evaluate either an advocacy/policy initiative or a program, decide together how and when you will use evaluation. Following are some common ways communities use evaluation.
- To gain insight (also called formative evaluation):
- Assess the level of community interest in a desired policy or program, and use that information to plan how to implement it.
- Identify challenges to and opportunities for a desired policy or program, and use that information to advocate for it.
- Confirm whether the data you are using are truly reflective of what community members are experiencing.
Learn more about how data can help you answer questions about the experiences and health outcomes of people in your community in our Why Use Data? Action Learning Guide.
- To improve a policy or program (also called process evaluation):
- Monitor the implementation of your selected policy or program and use the results to enhance components of the policy or program.
- Survey your target audience and use that information to improve the content and delivery of your communication, policy, or program.
- Assess staffing needs and internal systems (e.g. social media marketing skills, communication systems) to improve organizational capacity to deliver results.
- Assess the partnership to continuously improve and strengthen its capabilities.
- To support innovation in an uncertain environment (also called developmental evaluation):
- Reflect on feedback from an evaluator to help conceptualize, design, and test new strategies.
- Collect data about how a selected strategy is unfolding in real time in order to decide whether to abandon or continue in that direction.
- To evaluate policy or program effects (also called impact, results, or outcome evaluation):
- Measure the extent to which your outcome indicators are being met and use the results to improve your policy or program and be accountable to your funders.
- Use information about which target populations benefited most from your policy or program to target future efforts more effectively.
- Use outcomes to be accountable to your community and to your decision-makers.
As you work with your core team to clarify how and when evaluation will be used, consider:
- What do stakeholders want to know? How will they use the data?
- What does your community need and want to know? How will key leaders be engaged in shaping the evaluation plan and sharing results? How will you incorporate those who lack equitable access or opportunities, and those who are experiencing the worst conditions for good health, in the creation of your evaluation methods and processes?4
- What do funders require?
1. Office of Planning, Research and Evaluation,. Chapter 2: What is Program Evaluation? In: The Program Manager’s Guide to Evaluation. Second ed. Washington, DC: U.S. Department of Health and Human Services; 2010.
2. Frances Dunn Butterfoss. Evaluating Coalitions and Partnerships. In: Coalitions and Partnerships in Community Health. San Francisco, CA: Jossey-Bass; 2007.
3. Office of Planning, Research and Evaluation,. Chapter 3: Who Should Conduct Your Evaluation? . In: The Program Manager’s Guide to Evaluation. Second ed. Washington, DC: U.S. Department of Health and Human Services; 2010.
4. Sonali S. Balajee, et al. Equity and Empowerment Lens (Racial Justice Focus). In. Portland, OR: Multnomah County; 2012.