As 2020 made abundantly clear, changes and transitions can be difficult to manage. And while there’s no one-size-fits-all approach to adjusting to new circumstances, Change Matrix (CM) provides a diverse array of change-adaptive services and strategies that aid clients and partners in achieving their desired outcomes.
Perhaps one of the best primary approaches to facilitating change that CM offers is evaluation, which is the process of systematic and intentional inquiry and the use of data to inform decision-making around a specific organization, program, strategy, or initiative. At Change Matrix, we pride ourselves on our in-house approach of evaluation through an equity lens that centers the community experience.
That’s why this month, Change Matrix is highlighting staff Change Specialist and Evaluator Sandra Silva. As the director of the Leaders in Equitable Evaluation and Diversity (LEEAD) Program for the Expanding the Bench® (ETB), she champions the practice of culturally responsive and equitable evaluation (CREE), and is uniquely positioned to discuss why this approach makes for better evaluation.
What drew you to the field of evaluation?
Evaluation is very systematic and that’s how my brain naturally works. I think that once I joined my first evaluation project, I was really interested in developing an evaluation plan and thinking about, “What are the questions that we’re really trying to investigate here? What will be most useful?” There are a lot of decisions that you have to make. And then you have to structure a process to collect that information, analyze it or make sense of it, and then report it out.
I also really like talking to people and collecting information. I love doing interviews and focus groups, making sense of the project, and asking, “What are the learnings coming out of this work? How can this improve the program/project?”
How did Change Matrix come to adopt a CREE approach?
When Change Matrix began managing ETB, we needed to define how we were approaching evaluation. I read the literature surrounding equitable evaluation and culturally responsive evaluation and drew from the literature to identify the key elements that we thought were important to include. Then the team workshopped the definition over time with input from the ETB Advisory Team.
You’ve worked for decades as an evaluator with agencies that practice a more traditional evaluation approach. What would you say are the biggest differences between a CREE-focused approach and what is standard?
I would say that the traditional aspect or approach to evaluation is that the evaluator is the “expert.” So that person is making the key decisions around the evaluation, maybe with some input from the client. That approach really centers the evaluator and his or her expertise. At Change Matrix, when we go into a community for any sort of project, we approach with humility. We’re really there to learn. We’re not there to say that we are the experts, because we really want to engage people in the community as a part of the process so that the decisions and the inputs that shape the evaluation processes are informed by the people who are impacted by the work that is happening. We want to understand and honor the context and the history.
What is the first step in working with communities?
We have this exercise that we do when we first go into communities and we say something like, “What is a word that comes to mind when I say ‘evaluation?’” And most responses are negative and fear-based. I feel like our initial job is really to help people see the value of evaluation and see how they actually use evaluation processes every day in their lives but may not define it that way. We use a cooking exercise to walk people through how you do make a lot of these similar assessments in your life and don’t think of it like evaluation. We break people up into groups and have them each select a recipe. The process of walking through a recipe is much analogous to evaluation because you have to have the inputs, the ingredients, and as you’re talking through the “program” then you’re walking people through preparing a recipe.
How do you build trust with new communities that are being evaluated?
It begins with relationships and finding ways to connect with people. I think people can sense when you come from a place of humility and are open to understanding their experiences, learning about the history and context of a place, and hearing their concerns. I think this fear of evaluation in communities is well-founded—the feeling that evaluators have been extractive. And, again, if you think about the standard approach of “I’m the expert,” you often see evaluators and researchers that have gone into communities, collected information, and then disappeared. At Change Matrix, we work hard to establish how an evaluation might support community goals and establish processes that will inform evaluations that are of value to communities and can help drive change.
Learn more about Sandra Silva’s work online on Change Matrix’s website or by visiting ETB’s website to learn more about the LEEAD Program.
