Collective Introspection: The Seed of Good Monitoring & Evaluation (Part II)
KNUST, IDIN’s partner in Ghana, has been running Creative Capacity Building (CCB) trainings for artisans, farmers and other community members in and outside Kumasi. In these trainings, participants spend 3-5 days learning the design process, practicing basic hand tool skills, and then creating technologies to solve local challenges.
I was there to help KNUST staff develop a theory of change for their CCB trainings, so that they could define what success means for their program and then measure it. The staff decided that a CCB is successful if participants do the following:
- Continue to work collaboratively on their CCB technologies
- Create new technologies to solve other challenges
- Use these technologies for their own benefit
- Disseminate these technologies for the benefit of others
- Apply CCB ideas and skills to their lives
But what did the participants have to say about this? We wanted to know, so we interviewed 30 past participants in Suame Magazine, Konongo and New Longoro about the outcomes they had experienced after CCB. After several days of rich conversations, we had gained a few important insights:
1. Many participants were making new things from scratch. Many of the CCB participants were already makers by trade - welders, fabricators, seamstresses, and the like – and they had been trained to replicate a particular item. When we spoke to these participants, a common narrative emerged: before CCB, whenever a customer asked them to make something they had never made before, they would send them away. After CCB, they began to take on these requests. Many noted that they had developed the ability and confidence to to “make something new from scratch without a template” or that they had learned “a process to design any product.”
The number of new technologies they had created was impressive- participants had designed and sold tomato grinders, okra slicers, shoe shining machines, and helmets – all products that they had never considered making before, and all for paying customers. Fabricators were also improving their machines: for example, a welder specializing in gates used CCB ideas to develop a new locking mechanism. For fabricators who normally made and sold just one or two products, CCB represented an opportunity to increase their repertoire exponentially and improve their income.
2. The CCB projects themselves hadn't taken hold. We found that it was uncommon for groups to continue working on their CCB projects (cassava graters, rice threshers, and groundnut planters) after the training. Those who were not professional fabricators lacked the money, space or tools to continue working on the projects. The fabricators, on the other hand, cited lack of group interest or leadership to continue, and some mentioned that there was no market demand for the CCB product. Apart from the few farmers interviewed, the majority of participants did not report having used their CCB project for their own benefit, as the technologies created during CCB were mostly agricultural. For the participants, the CCB prototypes seemed to be a vehicle for learning only. In the meantime, KNUST has begun to engage local engineering students to evaluate and improve the CCB prototypes so they may someday reach users.
3. Participants were applying CCB skills and ideas to their lives. Most people cited examples of applying CCB skills and ideas to their lives, work-related or otherwise. For example, a seamstress had incorporated sketching into her dressmaking process, and a welder had begun asking his customers for feedback on his products. Some of these practices extended beyond the work sphere: a mother was better able to financially plan after learning the CCB step-by-step design process; a student used the design process to approach her exams: identifying where she went wrong and what she could do to improve. It was also common for people to mention that they were more comfortable working collaboratively and asking others for help, key ideas emphasized during CCB. We realized that participants were not only disseminating new technologies, but they were also disseminating and teaching others the CCB ideas and skills that they had learned.
Based on the above findings, the KNUST staff and I sat down to reflect and revise the theory of change and outcomes. Here is the final list of outcomes that we ended up with:
- Continue to work collaboratively on their CCB projects
- Create new technologies or improve existing ones
- Use these technologies for their own benefit
- Disseminate the technologies for the benefit of others
- Apply CCB skills and ideas
- Disseminate these ideas to others
KNUST’s monitoring and evaluation journey is just beginning. Now that these outcomes have been clearly defined and verified by CCB trainees, KNUST can start measuring them on a regular basis.
Had we leapt to the nitty-gritty of monitoring and evaluation before taking this time to reflect and verify, the data captured may have been totally off the mark, and we would have wasted a lot of time and effort.
Taking a few days to take a step back and reconnect with why we are implementing a program, whom we want to reach, what we want to change, and how to achieve this change could re-invigorate the program with purpose and launch a system to demonstrate how this purpose is manifesting itself on the ground.
Maya Ranganath is working as an IDIN M&E Fellow this summer. She just finished her Masters at the Fletcher School of Law and Diplomacy, focusing on Gender and M&E in International Development. Prior to Fletcher, Maya worked for the NGO Women in Cities International in Canada, and for J-PAL South Asia in India. She can be reached at email@example.com.