By Mrs Tamsin Greulich-Smith, Chief, Smart Health Leadership Centre
The funding of initiatives designed to improve the often ‘intangible’ aspects of well-being may not seek a traditional Return on Investment. Yet nonetheless, funding bodies may expect some indication that their investments have been worthwhile. Whilst we might instinctively feel that a project is ‘doing good’, evidencing this to investors or grant makers can prove challenging when the outcomes and impacts may not be easily captured using standard performance metrics.
This was the challenge presented to the NUS-ISS Smart Health Leadership Centre last year, by SG Enable, who are working on measuring the social impact of their grant-funded schemes.
SG Enable is an agency established by the Ministry of Social and Family Development dedicated to enable persons with disabilities to live full and rewarding lives. Apart from providing information and referrals for disability programmes, enhancing employability and employment options, and supporting those with disabilities to transition through different life stages, SG Enable is empowered to administer grants designed to assist persons with disabilities and their caregivers.
One of the grants SG Enable is responsible for administering is the Tote Board - Enabling Lives Initiative (“TB-ELI”) Grant. The TB-ELI Grant aims to fund disability-related projects to the value of SGD23 million over a five year period, through thematic grant calls. The Grant supports new, innovative and evidence-based projects in the following cross-cutting disability areas:
- Data and Technology;
- Caregiver and Natural Support; and
- Transition Management
Funded projects are expected to help improve various aspects of the lives of persons with disabilities and their caregivers, including better integration into society, greater empowerment, and improved sense of well-being. Whilst tangible tools and services will be funded in order to achieve such goals, measuring the impacts and outcomes of such initiatives is not so straightforward. Standard performance metrics are not deemed sufficient when considering the less tangible measurements of social outcomes such as integration, well-being and empowerment.
NUS-ISS has been appointed by SG Enable to develop a framework and a corresponding toolkit that can facilitate the measurement of social outcomes for each project funded by the TB-ELI Grant. The project has identified a suite of measures that can provide a more holistic measurement of project success, and a prototype of the measurement toolkit has been created for project managers to monitor the impacts and outcomes of the funded initiatives.
The measurement toolkit has been built upon a ‘Logic Model’ based framework, to measure each of the eight shared outcomes shown in Table 1. The framework is tailored to suit each individual project, to take account of variability in terms of:
- Nature of Project - Which could take the form of a programme, service provision and/or product development;
- Scope of Project - Considering aspects such as resources, tasks and deliverables;
- Scale of Project - Taking into account the size of the project, number of targeted beneficiaries; and
- Duration of Project - Length of project funding
To enable adoption and active utilisation of the framework, a toolkit is under development, comprising the tools of measurement for each of the eight shared outcomes in Table 1. These measurement tools are intended to be self-administered by Grant recipients or their project beneficiaries in measuring the outcomes of their projects.
Hence, principles of user-centric design underpins the development of the toolkit, ensuring that the toolkit is easy to administer, intuitive, with the data collected being straightforward to interpret.
Sally Khoo, Programme Manager for the Social Outcomes Measurement Framework initiative at SG Enable, explains what the framework and toolkit will mean for the agency.
How will the social outcomes measurement framework help SG Enable?
SG Enable: With this measurement framework, SG Enable, being the administrator of the TB-ELI grant will be able to help the grant recipients measure the success of their funded projects. This framework also lays the foundation to evaluate the collective impact of the entire TB-ELI grant on the sector.
What are your aspirations for the framework? What do you hope it might bring?
SG Enable: We hope to introduce to the sector an evidence-based approach in measuring and articulating the impact of their programmes and services. The Logic Model framework also helps grant recipients to design and plan service delivery with the beneficiaries and intended outcomes in mind.
Is anyone else measuring social outcomes this way in Singapore?
SG Enable: It is not uncommon to measure social outcomes of individual projects independently. From our knowledge, we might be pioneering this approach of using a standardised tool to measure pre-defined outcomes for different projects that are funded under one grant. Using a standardised tool will facilitate the consolidation of data for us to evaluate the collective impact of the TB-ELI grant.
Why does measuring social outcomes matter?
SG Enable: It is of increasing importance to measure the outcomes of social projects and their impact on peoples’ lives, which could serve as evidence of success to funders or stakeholders alike. It can also serve as a form of performance monitoring to continually review and refine the service offerings of the projects.
Why did SG Enable choose to work with NUS-ISS on this project?
SG Enable: Other than NUS-ISS’ domain expertise in the area of service analytics which includes programme evaluation and impact assessment, the team is also forthcoming in tailoring the framework to meet the sector’s needs.
Table 1. Shared Social Outcomes
Blending the disciplines, skills, and technologies required to make health systems smarter and future ready, the Smart Health Leadership Centre builds relevant capabilities through integrated training & mentorship, stimulates cross-industry collaboration and co-creation, and generates applied research-driven insights to advance opportunities for a smarter health future. Click here for more information