Introduction
As part of our work on the DDISA, OpenUp have been developing capacity-building resources for our beneficiary organisations. An important aspect of capacity-building is not just skills that help us improve our output, but also skills that help us build our institutions. In relation to the second area, OpenUp has been developing a bespoke grant management system for the DDISA project to help centralise the reporting information from our beneficiaries. But in doing so, we have been very cognisant of the questions of power, and knowledge, that underpin some of these activities.
The data we use to measure and communicate organisational impact have typically been developed, in the non-profit sector, as a tool for reporting to our funders. As constraints in the philanthropic funding sector compel us to begin reimagining our businesses – should we perhaps also be reconsidering our relationship to our own impact data, as a part of that organisational revolution? This paper explores what a more equitable and responsible impact data collection environment might look like for advancing African social impact activities.
This paper is primarily aimed at non-profits and social entrepreneurships as guidance in their impact measurement activities.
What is Impact Data?
The Oxford English dictionary gives two definitions of the word impact: ‘the action of one object coming forcibly into contact with another’ and ‘a marked effect or influence’. Impact measurement focuses on trying to establish that connection between action and effect. Impact measurement (and the data it generates) is connected to monitoring, evaluation, research and learning (MERL) practices that are typically used in funder reporting – though the former is more focused on what happened, and the second on tracking what is happening. So when we discuss impact measurement, we are thinking about the process of collecting and using data and information to understand how much progress your organisation has made toward its intended outcome.
The act of measuring your impact therefore helps you – quite simply – to:
1. Do good and know you’re doing good;
2. Improve your good impact;
3. Communicate your good impact; and
4. To do more good impact.
This is especially important as both funders and investors highlight the growing centrality of impact data for guiding their ultimate investment decisions, which ultimately speaks to impact data as a central component of your ultimate sustainability.
A challenge is that that measurement is generally just about your organisation, when increasingly we are trying to contribute to systemic change that involves a variety of actors. This is why researchers have posed a definition for social impact more broadly as “the result of a deliberate intervention by an individual, community, or organization with the objective of causing a positive effect on social, environmental, and economic well-being”.
The Politics of Funding
The reliance on external donor funds for operational activities can be a disempowering experience for non-profit organisations. Indigenous communities have – through considered research – helped to highlight the association between philanthropies and extraction as well, bringing to the fore not just the feeling of inequality as being based on needing donors, but also as a response to the practice of philanthropy itself.
The discussions about donor reporting in the past were chiefly framed as administrative frustrations for communities, but not necessarily in terms of how this relates to power. Different influences in the field of philanthropy, and from communities themselves, are now beginning to challenge this framing. Feminist grantmaking principles have importantly linked the reduction of administrative burdens on applicants in the practice of reporting with equaling out inequities between funder and grantee. Changing how organisations are required to report then becomes the solution to balancing out that inequality.
But there is a major underlying imbalance that is more difficult to solve because, as Mhlambi has specifically noted in the context of data governance:
“Consent without power leads to inequality”.
The choices of funded organisations in how they participate in the reporting process are very limited. But this isn’t meant to suggest that the very nature of sharing impact data with funders is unfair, or automatically extractive (after all, there is an organisational interest in letting funders know how much impact you may be creating with their funds). Instead, it merely seeks to highlight that the to and fro of impact data between grantees and funders must be understood with inequality front of mind, so that we can imagine how that very process might be adapted to be an empowering act instead. This exchange should centre shared value – which also means sending back learnings from funders, to grantees, which is not a practice that we see much of.
If impact reporting can centre shared learning for both grantees and funders this becomes part of the solution. When we consider learning, though, a further question emerges: why are we not able to receive more collective community intelligence? What learnings are we getting back about our communities as a whole, from all our partners, so that we can take action collectively?
The Power of Your Data and Your Story
The exchange of information and data can be disempowering and extractive. Simultaneously, the sharing of data and information can be an act of agency and control – actively helping us shape a narrative, and give voice to the communities we want to help, making their needs and stories visible.
Yet the inequality lens also raises an important spectre on impact reporting – when your impact reporting is chiefly driven by the needs of your funders, you begin to “submerge and elevate” different kinds of data based on their needs that could be in opposition to the desires of the communities your data is meant to represent. Because – as change makers – we also extract data from the communities on whose behalf we are acting, and therefore also hold power (and responsibility) in relation to that role.
In this regard, feminist-driven data principles have highlighted that how we manage data we take, and how we communicate that data, must be driven by inclusion and fairness and – in OpenUp’s own developments of these principles – radical honesty. Implicit, though sometimes not highlighted sufficiently, is that the act of generating and producing quality impact data is an act of responsibility born by us given the privileged position we hold as data processors (and storytellers).
Organisational Data Sovereignty
Your organisation, similarly to governments, is a data holder – and to act with sovereignty i.e. to act without coercion, requires us to try to ensure our data practice is also sovereign. Data sovereignty isn’t necessarily about who ‘owns data’, but is often better understood by being able to understand who ‘controls’ (and extracts value and meaning from) the data we are concerned with. Data sovereignty does not exist without the infrastructure capacities required to benefit from that data – and most benefit in data is realised by its use, not simply its “holding”.
This also shows us that, even if we have data infrastructure that is independent – like our own servers, or cloud servers we know have adequate contractual protections – if we do not have the internal skills and processes that allow us to interpret that data meaningfully, and tell its story, we cannot demonstrate data sovereignty. This means developing both processes that allow you to process data, but also investing in systems (like software) that better allow you to analyse, and make meaning, from it.
For example, OpenUp (in partnership with other organisations, but also under the DDISA in particular) have been working on open source grant management software that allows people to report, and track progress, all in one place. In the longer term – and given the collective intelligence issues raised – it will be worth considering how to use open source software as a foundation for collective infrastructure as well, helping to reduce cost and maintenance burdens. Shared infrastructure can be an opportunity not just for sharing report information one way – but for exchange. In our role as “funder” partner in the DDISA, we are trying to reimagine this software as an opportunity in the longer term for also providing data back to our beneficiaries in a way they can use for their impact in the longer term.
But these needs present organisations with a significant bind. As our early surveys with our DDISA beneficiaries demonstrated, challenges exist both in terms of the financial resources required to invest in datafication and digitalisation, but also (somewhat related) not having sufficient human resources for data and digital functions. Organisations will need to transform their data governance environment as a first step toward improving their output of quality impact data – but, this investment may be worthwhile given how that data can itself contribute to organisational sustainability.
Data sovereignty also requires political sovereignty – which, at the level of organisations, may be best described as sovereignty in terms of the decisions that are taken. An organisation that cannot determine its own strategy will not be able to exert true sovereignty in implementing that strategy, regardless of what it is. This is why, in Diagram A, we have placed such some emphasis on the determination of the “data agenda” in identifying where a power imbalance in relation to data may exist.
Collecting and Structuring Impact Data
For the purposes of this brief, we will not try to unpack how to begin transforming your data governance systems in their entirety (there are, however, some useful OpenUp guides on building your broader data skills at our DDISA School for when you do have time housed here). But in considering how to transform your approach, and your position, on impact data in particular, it is worth first unpacking the peculiarities of that data.
Typically, you define your impact in terms of your organisation’s own theory of change – as this frames your ultimate goals, which is the final measure you are aiming for. This links to the more traditional experiences organisations may have had in developing MERL frameworks as part of kicking off of new programmes and projects. It is however also useful in constructing impact frameworks to link them to existing frameworks that already exist – this makes it easier for you to translate your story in terms of metrics and measures others already understand, if you can adapt them adequately to align with your own measures. Next Generation Consultants have provided a useful table on current impact frameworks of relevance to the South African social impact community in particular (though our regional partners can also use them for guidance) as:
| Framework | What It Measures | SA Relevance |
| IRIS+ (GIIN) | Standardised impact metrics for investors; 5 Dimensions of Impact (What, Who, How Much, Contribution, Risk) | Growing SA impact investing market; DBSA, IDC, NEF increasingly require IRIS+ or equivalent alignment |
| SROI (Social Value UK) | Monetises social value created relative to investment; stakeholder-led; ratio of impact to cost | NPOs and intermediaries can use SROI to demonstrate value to CSI managers and government funders |
| GRI Standards | Sustainability reporting on environmental, social and governance performance; internationally recognised | JSE-listed companies increasingly require GRI; CSI funders and their NPO partners must align for reporting |
| SDG Alignment | Tracks contribution to the 17 Sustainable Development Goals and 169 sub-targets | Government and bilateral donors require SDG alignment; AI significantly reduces the manual mapping burden |
| Impact Management Project (5D) (Impact Frontiers) | Structured evidence across What, Who, How Much, Contribution, and Risk dimensions | Adopted by leading SA foundations and impact investors; increasingly referenced in grantee reporting requirements |
| B.world (AI-Native IMM) | AI-powered impact design, data collection, analysis and storytelling platform built for social organisations | Emerging platform; intermediaries could deploy as shared MEL infrastructure for grantee portfolios |
| 60 Decibels / Lean Data | Rapid beneficiary feedback system; captures community voice at scale through standardised mobile surveys | Currently underused in SA social sector; AI makes community voice collection and synthesis significantly more feasible |
Once you have selected a framework more broadly, and what you want to measure, you need to consider the indicators and data sources for those measures. Those who have engaged with funders will have no doubt been exposed to conversations around “outputs versus outcomes”. In OpenUp’s own MERL frameworks, we have used this simple table to help us distinguish between the key different areas which we measure as:
| Activity | Output | Outcome | |
| Definition | What we do | What we produce | What we achieve |
| Question | What happened? | How well did it work? | What was achieved? |
Ultimately though, impact moves beyond outcomes. Whilst an outcome is a finite and often measurable change, comparatively impact is a far broader concern:
“While the outcome evaluation tells us what kind of change has occurred, an impact evaluation paints a picture as to how a program might have affected participants’ lives on a broader scale”.
As you move from activity to outcome to impact you begin to see what one of the most significant challenges for impact measurement is – and that’s establishing the clear nexus between what you have done, to the change you have seen. However, you can establish that nexus using quantitative and qualitative data that – with sufficient empathy – you can translate to a compelling impact story about your work. Ensuring you are able to collect, and translate, both quantitative and qualitative data, will improve your capacities for impact storytelling.
At this point, it is worth reflecting specifically on lean data methods. These data methods focus on creating “downward accountability” in the selection of metrics, which can be seen as consistent with the idealised impact data flows (and influences) outlined in Diagram A. This idea comes more squarely from the world of social entrepreneurships, and focuses on simplifying the data collection practices, using digital tools, to focus impact data collection around a central impact question – where that question is not about what is required in reporting, but rather answering what measure will best allow an organisation to learn and adapt to better serve its communities (or impact goals). This kind of approach is useful because it ties impact data collection and analysis more directly to overall organisational strength for improving impact – making it more directly connected to its ultimate sustainability, and therefore making it a more justifiable (and cost effective) investment, whilst also emphasising “leanness”.
Once data has been consolidated, it may be worthwhile emphasising in the consolidation and analysis the following:
– The change created: focusing on the “before” and “after” of your activities with quantifiable measures helps to provide evidence that supports your approach, also justifying it as an investment case. Also emphasising human examples is this “before” and “after” is useful given a more comprehensible demonstration of your successful theory of action in practice.
– The connection between the individual and the systemic: demonstrating how individual successes feed into broader systemic change is important for both contextualising why an activity is a contributor to more comprehensive change, and can also act as support for replicating and/or scaling of activities.
– Sustainability and growth: demonstrating potential sustainability pathways is its own description of positive impact, because it evidences a more feasible impact pathway in the longer term. But this is not necessarily always possible, and those stories without a sustainability pathway are not less valuable – rather, it is just an advisable point of emphasis when in existence.
– Emotional and logical appeal: try and craft narratives that engage both the heart and the head, combining human stories with strong, indicative data. This of course is a more communications-focused piece of advice, but can help maximise discrete pools of data.
Putting Data Sovereignty into Impact Practice
We believe that improving your impact data processes is both an opportunity to improve your sustainability, but also your own sovereignty and autonomy more broadly. In our work on general data readiness with organisations, we have done work around the “9 dimensions” of data readiness (see Workshop 2 in our DDISA School materials), which have been simplified into the following three overarching areas:
1. Plan and protect (strategy, governance, security);
2. Collect and keep (collection, storage, technology and systems); and
3. Use and grow (reporting, AI and emerging tech, literacy and culture).
Considering those three areas, we can apply some specific learnings for impact data in particular as follows:
Gather data with purpose
– Rooted in responsible data practices, gather the data that is necessary to demonstrate your impact. Ensure this is connected to your organisational strategy.
– Be guided by your organisation’s goals, but borrow from frameworks that will also help your data talk to a broad audience of readers.
– Part of your responsibility as a custodian of data is ensuring the quality of that data – and its ability to honestly represent what you say it represents. It is also ensuring the security of the data, particularly when it relates to beneficiary data.
Exert control over data
– Invest in the right infrastructure for controlling and analysing your data, which can also simply mean investing in the design of simple processes that allow you to govern your data.
– Building internal skills for processing and interpreting data is a vital step for ensuring your data sovereignty.
– Data governance – which means constructing rules and standards around your organisational data – are also an important opportunity for embedding values like fairness, inclusivity and representativity into their very collection.
– Data governance is also an opportunity for building trust into your data from outside actors, whether it be the communities you serve or the funders you report to.
Convey that data meaningfully and grow
– Acknowledge that how you communicate your data matters (a recent internal study by the World Bank revealed that 31% of its reports were never downloaded, and 80% were never cited. Of the reports that were opened, people read at most around 20–28% of the content. Pdfs are where data goes to die).
– Use the same data, but to tell stories in different ways. Different audiences will be looking for different things in relation to your impact – and being empathetic to that will improve the likelihood it’s ultimately valuable to the community at large.
– Charts and visuals communicate instantly what text takes sentences to explain.
– Standardisation, which also means collaborating with a variety of groups in how you think about data, is a way we can increase the pool of useful data we can all learn from.
A Collective Intelligence Agenda
Whilst this paper has significantly focused on individual organisational action steps that can be taken, there may also be a broader collective consensus worth discussing for responsible data custodians. When we have discussed the importance of building your internal capacities to grow your data pools (and data projects), implicit in those strategies is the potential of data sharing and data exchange outside your organisation for that to be achieved.
Collective data pools help us to identify broader patterns. Sharing data between organisations can help us:
– Interpret our own data in a better by providing us with more information about its true context;
– Supplement your own data where there are gaps;
– Identify new learnings we may not have designed data collection for when we create our own data strategies; and
– Expand our data pools without creating more burdens on our beneficiary communities.
If we can share data, we can also potentially share infrastructure – reducing the costs that are proving to be such a digital blocker. This infrastructure would require us to agree on data standards, though not necessarily in a formal, once-off way. And there is another often underestimated organisational benefit to sharing data – the more lean, and controllable, your own data collection and processing practices are, the less the risks that exist (i.e. in relation to data protection, cybersecurity, quality control, etc.). Collective data sharing can therefore allow you to reduce your own risks by sharing the risks a bit more across organisations as you scale your data pool without having to scale your own data activities. However, for that benefit to accrue it means we don’t just need to be sure about our data practices, but that of our partners as well.
In other words, ultimately creating a collective intelligence ecosystem for impact data will rely on organisations first ensuring they have in place their own responsible data practices (which is why we foregrounded organisational data practices first). To create the right trust environment that allows organisations to share will require data governance to be a practice of each role player, and then to have a data governance strategy for the data sharing activities themselves given that each roleplayer is an existing data custodian.
There are a variety of different opportunities for advancing this collective agenda. As seen above, using shared software can be a shortcut toward creating standardised, shareable data amongst organisations. Github repositories of de-identified impact data can be established, and promoted, amongst existing social impact networks and communities. And there may even be opportunities to lobby current and past funders for better, or renewed, access to already reported data. There are a variety of avenues, therefore, for moving toward a more collective sovereign impact data agenda – but this must be founded on embedded responsible data practice.
Conclusion
This paper proposes that – for non-profit organisations and other social impact organisations – generating our MERL and impact data has often been driven by unequal relationships between organisations and their funders. Impact data may be reframed both as a way to empower our own organisational intelligence, but also as a way to contribute to collective and systematic impact (through collective intelligence).
To do that, however, requires us to build our own data sovereignty and governance, not least of all through building our data skills. We need to improve the systems – and opportunities – we have for sharing learnings through impact data, and be committed to the quality and ethics of that data. And whilst we start with our own organisations, ultimately this needs to be a collective activity for driving regional impact together.
Quite simply, our inside shapes our outside. To create a fairer data environment requires us to take charge at an organisational level for our own responsible data practice first – ultimately being guided by our own, measured vision for creating a more positive world. We need to build data power for ourselves, for our partners and for our ultimate beneficiaries.
This blog was prepared by OpenUp as part of our activities under the Digital Democracy Innovation in Sub-Saharan Africa Fund.