This is the last in series of 6 posts were we explore 5 Evaluative Practices, that if adopted put you on the path to moving beyond conducting evaluations but to fully embrace a focus on impact as a means to guide your decisions and actions. Our last post highlighted “differentiating what you want to know, can know and need to know.” Key take-aways from that post were:

• Knowing how to ask the right questions at the right time (See video)
• Distinguishing between the information needed to determine if your efforts are progressing as intended, and having the impact desired.

Evaluation is not inexpensive but often it is more expensive (in many ways) than it has to be because there lacks an organizational culture of engaging in focused and relative inquiry, analysis and decision-making guided by data all of which is intended to increase and/or maintain intended impact.

The consequence of this is that when information is needed (for internal or external accountability) the process of honing in on the most critical questions about program delivery and impact is a more challenging and stress inducing undertaking. By building inquiry in to the practices of the organization, it will be easier and likely less costly to engage in “evaluation” because as an organization you will have processes and even more importantly practices in place to support.

Here are some ideas of things you can do and which our clients do already:

1. Incorporate Reflective Practice in to Supervision – Thinking about how, why and to what end is a deliberate mindset. It is a new muscle that needs to be trained. How are you using supervision to introduce and support this type of thinking so it is not something someone else does but something everyone does?
2. Structure Staff Meetings Differently– Build in to the beginning of each staff meeting an opportunity to reflect on practice (a program that just ended or even better just started) with a set of standard questions allows individual and group thinking. Themes that emerge should inform next steps with regard to programmatic changes and/or messaging to others about areas of success.
3. Convene an Internal Evaluation Work Group (EWG)– This is a group that spans the organization charged with thinking about the most critical questions to answer about how efforts are being implemented and to what end. They can lead the thinking in determining the ways in which existing data collection and analysis efforts (i.e., intake forms, workshop assessments) are useful and make recommendations for changes.
4. Develop an Inquiry Matrix – Create a table (using excel or word) that identifies the most critical questions your organization wants to answer about its efforts (first column), cross referencing with the existing data collection tools (top row) and the degree to which what you currently have in place answers those questions (we often use a 1 to 3 scale of 1 being the tool really address the question and 3 not so much). This simple analysis will give you an inventory of sorts of your existing data collection and its meaningfulness.
5. Develop a Data Collection and Analysis schedule - Collecting the information is just step one. It has to be analyzed and then used. Just like you do with your programs, think of inquiry, analysis and decision-making as a program. Make a schedule. Identify who will analyze the information and when it is needed to make a decision. This will help you manage resources and to systematize the process. Use your EWG to guide this process.

These are just 5 ways you can move to becoming more systematic. They only cost is your prioritization of staff time and a commitment to shift your culture. What are some other ideas and strategies?

 This is the 5th in a series of posts were we explore one of 5 Evaluative Practices, that if adopted put you on the path to being evaluative. The last post highlighted how important it is to Leverage and Link Your Efforts so you are designing efforts that maximize resources and have the greatest likelihood of success. Take-aways from that post are the following:

• Recognize that your organization is part of an environmental landscape and understanding where you fit within that and your unique contribution can be helpful in narrowing in on the outcomes you hope to accomplish.
• Outcomes don’t happen by magic at the programmatic or organizational level. They occur because one has thought through a pathway of activities/strategies, relationships, interactions and changes that are most likely to lead to the desired end.

The next critical practice is to “differentiating what you want to know, can know and need to know.” Once you get to this point, you have adopted the idea that information is important and it’s not all what you can see and feel but also what you can demonstrate. This is all about knowing how to ask the right questions at the right time .

As you continue on this journey, distinguish between the information needed to determine if your efforts are 1) progressing as intended, and 2) having the impact desired.

Getting clear on what information (guided by the most relevant questions) is most salient to your work and intention and being comfortable with what you can know (based on available data, time frame, resources, appropriateness, etc.) is an evolutionary process. Like sport you wish to master, it takes time and
training for the muscles to respond and behave in the way you wish. Part of that journey is knowing what equipment best suit your style and preference and yields the desired results as well as determining a training schedule you can stick to (more on that in our final post).

Some tips to pave the way include:

• Understand where your organization or efforts sits in terms of its overall development
• Know the difference between process and outcome questions and at what stage in your development it makes the most sense to ask what
• Look to your colleagues who engage in similar work or efforts and ask if they might share their thinking, internal working documents including logic models, evaluation frameworks and measurement tools.

This is the 4th in a series of posts were we explore one of 5 Evaluative Practices that if adopted put you on the path to being evaluative. The last post highlighted how important it is to Differentiate What You Know from the Difference You Make and how that can open up a set of possibilities. Core to that post are the following:

• Getting clear about why you do what you do and the “to what end” is a new muscle in the non-profit sector which needs exercise.
• Once an organization gets clear, it can be a transformative event that opens up possibilities, focuses efforts and strengthens evaluation efforts

The next critical practice is to “leverage and link your efforts.” Important in both small and large organizations. For large organizations, because it is easy for the pieces and programs to get away from the core purpose and intended impact of your org. For small organizations because there is very little cushion and thus everything you do must matter. This includes not just thinking about efforts internal to your organization but also how you relate to others whose work supports and compliments your own.

In our own practice, we love this part of the process which often is iterative with Practice 2 (Differentiate What you Do with The Difference You Make). If we are doing program /initiative level work, it starts as simply with a group of staff people each armed with a set of color coded large post-its (one color each for outcomes and activities) and a blank wall. Participants are given the opportunity to brainstorm outcomes and activities which lead to them. We then move the pieces around the wall, the dialogue begins and the opportunity for clarity and alignment happen through exploration of the following questions: What leads to what? What is missing? Is that the right outcome? What can we really accomplish? What else do we need to do to reach the outcome? Can we do it? Should someone else? How do we connect?

For more complex endeavors, conceptual models such as network analysis and system mapping can be helpful. And logic models and theories of change are additional tools to assist you in making connections and getting clarity on how the pieces work in service of each other and where your organization fits in the larger environmental context.

None of our work happens in a vacuum, and for us to have true impact as well as understand the degree of impact we CAN have, it’s important to contextualize, be explicit about intention, and develop and implement programs/projects/efforts which have the greatest likihood of success. All of which move you along the path to being evaluative.

What tools have been most helpful to you? What have been the results of leveraging and linking your efforts in your organization?