Spring 2023: Impact Evaluation with an Equity Lens

Impact Evaluation with an Equity Lens

We believe that the people most impacted by a program or organization are best positioned to assess and measure impact, so these are the people we elevate in our work. – Jessica M. Payne, PhD, Broadsight Founder and Lead Consultant

With Broadsight, equity is a core mindset and pillar of connection and understanding

Broadsight centralizes equity by selecting clients with a demonstrated commitment to expanding access to opportunities and resources for all people regardless of background and identity. We seek out client partners who lead efforts to ensure that people who have been excluded get included.  As we collaborate, we’re always asking questions.

• How important is equity in our clients’ mission? In their sector? In their systems change initiatives, innovative approaches, and entrepreneurial ventures? 
• What is the connection between client ‘mission’ and actions? Who gets impacted most, and how are clients addressing remaining gaps? 
• How are clients’ approaches inclusive, collaborative, community-initiated, top down?
• Whose voices do clients activate, elevate, centralize and how?  Whose voices do clients still need to include and integrate?
• What does clients’ available and emerging data capture? What data do clients still need to capture and how/when will clients address data gaps?
• What is Broadsight’s role in facilitating inclusive practices? Addressing gaps?

Impact evaluation with an equity lens focuses on distribution of engagement and ownership.
Read more below in Musings: What is impact evaluation and what role does equity play in Broadsight’s work?

Client Spotlight : Venture Well 

Venture Well engages Broadsight to access equity-centered strategies to improve undergraduate student success

As a facilitator of national and global innovation and entrepreneurship initiatives, VentureWell (Hadley, MA) has engaged Broadsight since 2014 to evaluate their programs and partnerships (originally as Jessica Payne Consulting). 
VentureWell partners regularly on innovation initiatives associated with the Bill and Melinda Gates Foundation’s (BMGF),  including the Foundation’s Frontier Set initiative. From 2017-2022, the Frontier Set engaged select U.S. colleges and universities, state systems, and supporting organizations (roughly 30) that committed to take steps over a 5-year period to increase undergraduate student success. Specifically, Frontier Set sought to reverse attainment gaps for select students, including Black, Indigenous and People of Color (BIPOC), first-generation, and socio-economically disadvantaged students. To accomplish this goal, BMGF, with the help of its partners, including VentureWell and Broadsight, developed a networked learning structure for its members to explore and learn together about strategies to transform towards more equitable student success.

The BMGF wanted to know which strategies had the greatest impact in catalyzing improved success among the select student populations. Broadsight’s contribution was to extrapolate learnings taking place on individual campuses and detect broader patterns across the larger Frontier Set cohort so as to better understand the key drivers for the institutional transformation needed to close achievement gaps. Our team’s analysis of site reports on campus activities and strategies informed VentureWell’s approach to campus engagement, and was foregrounded in BMGF’s report, Frontier Set: Working Together for Equitable Student Outcomes. A final year study by our team highlighted the crucial role of leadership commitment and infrastructure development to support equitable access and opportunity for each student.

The shift from transactional to a deeper, transformational concept of equity in this work was an emergent process, highlighting areas to further align core mission and equitable practices. Though the Frontier Set and some member sites were talking about equity in Broadsight’s first round of research in 2018, the overall initiative was still in early stages of conceptualizing equity relative to institutions and their student success strategies. 

Reflecting broader national reckonings around systemic racial oppression, Broadsight’s second round of work in 2021 contributed to efforts to elevate equity as a theme and a mindset in Frontier Set convenings, webinars, and site engagement. As with any systems change effort, the work continued to illuminate areas for ongoing engagement. As but one example: In a project that tended to privilege leaders and mid-level staff as drivers of institutional change, the lived experiences and engagement of first gen, low-income, and students of color at the center of the BMGF call to change were often difficult to detect.  

From that most “on the ground” student level, to the broader task to dismantle entrenched structural and systemic oppression in educational institutions, our Frontier Set experience affirmed that truly equitable transformation is most likely to emerge through steady, facilitated processes that engage leaders, faculty, staff, and students (we would also add alumni, parents, and guardians) to face up to, re-vision, and rebuild our foundations. From there it may be more feasible to align an equity-centered educational mission to sustainable, equity-centered practices. We may then begin to more readily see bold equitable outcomes that close success gaps.

Wherever our clients are in their equity work, we strive to cultivate inclusive mindsets that draw more people to the table where new possibilities get envisioned. Because we all gain wisdom by hearing more voices than we may have listened to before. 

Learn about Broadsight’s work on the Frontier Set.

News

“Wilkerson brings the invisible and marginalized into the light and into our hearts”. -bio of Isabel Wilkerson

by Caroline Gurek, Impact Data Analyst & Operations Associate

Isabel Wilkerson is an incredible researcher, oral historian, journalist, author, and speaker set on illuminating a past and system we have inherited. She is the recipient of the Pulitzer Prize and the National Humanities medal and is the author of two non-fiction books. Her debut book, The Warmth of Other Suns: The Epic Story of America’s Great Migration, used the life stories of three individuals to tell the larger story of how 6 million Black Americans moved North and West to escape Jim Crow laws.

Her second book, Caste: The Origins of Our Discontents, delves

deeply into the history of how a caste system to divide and rank human beings developed and survived in the United States. Starting with the arrival of enslaved Africans in colonial Virginia in 1619 to the Presidential election of 2018, Wilkerson illuminates the workings of the caste system as interwoven into the country’s past, present and future. Comparing the American caste system to similar structures enforced in Nazi Germany and India, Wilkerson examines the mindsets and societal structures that positioned enslaved Africans and their descendents through the present day as the “Untouchables” of the U.S. 

Arguing that U.S. racism and caste are not synonymous but deeply related, she traces how White citizens of the past selected themselves as the highest class and ranked others below them. Even though there is no one alive today who actively engaged in legal enslavement, Wilkerson suggests that all Americans need to grapple with these issues because of the ways they continue to condition and structure an American society that privileges the contributions and aspirations of certain castes over others. Wilkerson uses metaphors, imagery and analogies to describe the caste system in an accessible way. She compares American history to an old house–when you inherit a house, the foundation may be cracked and it might need a fresh coat of paint but it is still your responsibility to repair and maintain the house. 

 “Not one of us was here when this house was built. Our immediate ancestors may have had nothing to do with it…We are the heirs to whatever is right or wrong with it. We did not erect the uneven uneven pillars or joists, but they are ours to deal with now”. 

When thinking about evaluation and equity, it is imperative to recognize the systems of oppression and elements of caste that will be present in any project. In many ways, methods of evaluation have perpetuated inequities because of how and what data gets collected and who interprets it. Communities, people and stories will be left out of evaluations without equitably designed and implemented approaches to the work. 

At Broadsight, we try to continually educate ourselves and center equity, ownership, wellbeing and the human experience in all that we do. This is an ever evolving practice to be continually re-visioned and integrated into our work with clients. Isabel Wilkerson sheds light on the structural changes and transformative mindsets needed to center equity as a necessary human dimension of life in the United States.
​We highly recommend Caste as well as other works by Isabel Wilkerson.  

To learn more, here is Isabel Wilkerson’s website

To learn more about Broadsight’s approach, look here. 

Musings: Impact Evaluation with an Equity Lens

Jessica M. Payne, PhD, Broadsight Founder and Lead Consultant  

We do impact evaluation to help organizations understand how their programs affect individuals they seek to serve. Most of our clients have solid track records, but they may not yet know about how their latest programs and strategies are actually working. As change makers who experiment with new approaches, they’re especially accountable for demonstrating their value and that their innovations are worth the staff, time, and finances they cost. Impact evaluation helps signify this value by measuring the impact or outcomes that result from a particular program or strategy. Impact evaluation is a means to generate evidence of success and of value.

Equity and equitable impact evaluation practices are especially important in the innovation spaces where Broadsight works. 

Impact evaluation demonstrates the efficacy of an intervention or set of strategies in ways that benefit internal systems and external persuasion. Institutions expend considerable funds on programs and strategies without knowing whether they actually have intended impact. Evidence obtained through an impact evaluation helps practitioners communicate accurately about the benefits of integrating new strategies into established systems and programs. Practitioners may use results of an impact evaluation to build new external partnerships and funding streams to sustain continued use and testing. Data-informed knowledge about how new strategies work and outcomes they promote allows practitioners to talk more precisely about intentional patterns of change. For effective work with internal teams and external constituents, partners and champions, clarity about how to instigate desired change is a crucial part of leveraging buy-in, engagement, and funding.

An Equity lens informs how we collect information and from whom. We believe that the people most impacted by a program or organization are best positioned to assess and measure impact, so these are the people we elevate in our evaluations. While organizational staff and boards are crucial, we typically select our clients because of the ways they centralize engagement and ownership among their community stakeholders. Who are the key stakeholders whose perspectives and experiences can most effectively guide decisions about growth and change? What mechanisms are in place to gain community feedback and is that an ‘extractive’ process or does it reflect actual ownership stakes in decision making and other strategies and outcomes? These questions inform our selection of clients we wish to work with – because we want to collaborate with people who use equity as a mindset and a pillar of their work to make their communities and our collective world a place where all individuals can thrive. 

One of the things we hear most often from our clients is that the work we do together gives them greater confidence when pitching to funders or engaging with community members to get them involved. We’re proud of how our clients seek to build equitable worlds and grateful to be a part of their journey. 

Learn about Broadsight’s Impact. Learn about our clients’ journeys with Broadsight.

Gratitude 

We’re grateful for the return of spring, renewing old connections and making fresh alliances. Here are a few things we’ve been doing.

We attended the opening reception for the stellar exhibition, Nelson Stevens: Color Rapping at the Springfield Museums at the end of March (showing until September). We were glad to see the great community turn out for this event, and pleased to support the memory of Mr. Stevens and his family who were in attendance. A shout out to Common Wealth Murals and director Britt Ruhe who played important roles in bringing attention to Mr. Stevens’ artistry and activism through the 2022 recreation of two of his murals in Springfield’s Mason Square neighborhood.

Jessica was thrilled to reconnect with Erika Larrieu, a professional coach who is helping to gather a team of practitioners to develop transformational resources for young adult career building, mental health and resilience. Central in this effort is Helen Horyza, creator of Elevate Your Career, a validated assessment tool with associated YouTube channel, program, and book. We hope to get more involved and will share information as our conversations continue. For now, thank you Erika and your beautiful family for hosting and hiking us in such an exhilarating way!💗  

We benefit from living within the Five College Consortium which attracts world renowned influencers like Ta-Nihisi Coates, who spoke at Smith College in early May. Coates shared reflections on his current engagement teaching writing at Howard University. He reiterated the imperative to reckon with genocide and enslavement as defining features of our collective national story and identity. A theme of tending to one’s own innate passion – that thing you wouldn’t want to and cannot live without – and the calling of one’s authentic voice resonated in his comments on writing and the pull to speak his truth.

Broadsight is a proud founding sponsor of Secret Planet. You may be familiar with community supported agriculture or CSAs – a strategy of using purchased farm shares to invest in and obtain healthy, fresh food from local farmers. One of our region’s music innovators, Edo Mor applied this concept to music performance. Based in Western MA, Secret Planet shareholders gain access to live performances and in turn support efforts to bring live acts to the region from across the globe.  As a team made up of avid music consumers and as a business committed to expanding connections and communion, Broadsight wanted to support this endeavor from the get go. Learn more about Edo and the genesis of Secret Planet here!

Thanks to dear colleagues for contributions and support in the creation of our newsletter. We’re switching from monthly to quarterly newsletter production, so look out for the next newsletter on Resilience in the summer months.

Cheers for Springtime!!!

– The Broadsight Team – Jessica, Cinamon, Ariana, and Caroline
Check out Broadsight’s previous newsletters.