While carrying out research on evaluation methods used in adult education projects targeting antidiscrimination, we found out that most of organisations and professionals working in the field are carrying out evaluation during and right after their projects, and it soon became apparent that methods measuring long-term effects are seriously missing.
While searching for such techniques, we found the Most Significant Change method (MSC), which was originally invented to evaluate a complex, participatory, rural development program in Bangladesh that came to be used by many international development organizations. It represented a radical departure from the conventional monitoring against quantitative indicators commonly seen in this sector, as MSC involves regular collection and participatory interpretation of “stories” about change rather than predetermined quantitative indicators. The stories of participants are collected along a question, for example: ‘During the last month, in your opinion, what was the most significant change that took place in the program?’Thus, MSC is based on the subjective stories of participants and stakeholders, and strongly builds upon the dialogue between these actors. We believe that MSC can be adapted to evaluate long-term effects of non-formal education projects focusing on antidiscrimination.
In the setting of international development work, MSC is organised in a complex system of multiple levels, but in our field we propose a simplified, adapted structure of the following steps:
The collection, selection and discussion of the stories can be done in a collaborative way. The selection phase can be also preceded by a discussion about what are the criteria of a significant story (what makes a story more important?), to help the group make a decision, which may also take the form of voting for stories.
Once the most significant story (or several of them) is selected, it can be shared and explored in more depth. This can mean that the owner explains the story in more detail to the group, followed by a discussion. The story can also be shared publicly, within a professional, or a wider general audience.
The MSC method provides a structure that can be combined with creative techniques, for example it has been used in combination with Participatory Video by Insight Share (https://insightshare.org/baseline-midline-endline-a-hands-on-pvmsc-experience/).
We would like to encourage facilitators to use this framework creatively, and adapt it to their audiences and methodologies to come up with meaningful and creative ways to evaluate the long-term effects of their projects.