Evaluation: Quality & impact
Are we doing the right thing, and are we doing it well? In order to constantly improve performance, medica mondiale regularly poses and answers these questions. Independent evaluations and the integration of insights gained from them help us to ensure effective and sustainable support for women and girls in war and crisis zones.
The continuous monitoring of the effectiveness of our work through external evaluations is intended to reassure our staff that we do indeed provide high quality support. Our sponsors and donors rely on us to ensure that our commitment to women and girls has a long-term effect. These women, who experience the violence, also trust us. So there are many good reasons for setting demanding standards for medica mondiale's daily work.
“The involved women are empowered both economically and psychologically. They are taking care of themselves, have opened a shop to sell their manufactured products, generate income and are accepted by their families.”
Independent experts are commissioned by medica mondiale to determine how effective, meaningful and sustainable the projects and programmes abroad are. As the basis for external and internal evaluations, medica mondiale takes the quality standards of DeGEval (Germany Evaluation Society). A systematic evaluation and assessment highlight potentially weak areas and supports learning processes.
Some typical questions about project work:
- Do we reach our targets and goals?
- Do we improve the situation of the women and girls permanently?
- Do we work effectively?
- Can we be even more effective with the same effort and resources?
- Is the work being carefully documented and evaluated, to enable the responsible project leaders to correct mistakes quickly?
- How far do we implement our basic principles and quality features?
“The support provided to the women is a lifeline and lives are saved and changed with the provision of support by MA.”
The results of the assessment will be included in an Evaluation Report. This report should identify strengths and weaknesses and make recommendations for improving the quality of projects. Using this basic information, local colleagues and responsible medica mondiale project leaders will discuss how best to implement the recommendations.
In addition to the evaluation of overseas projects, medica mondiale also puts great emphasis on the regular examination of other areas of work.
Examples of further areas of activities to be analysed:
- Advanced training courses on war-induced trauma, held in Cologne
We are continuously extending the evaluation processes, as they are a necessary reminder and confirmation for us that we are, and remain on the right path.
“medica mondiale sees itself as a ‘learning organisation’.
Our expertise is gained from the project work and evaluations
help us to continue developing this.”
Decades of experience working in war and crisis regions has led medica mondiale to develop a variety of operating methods and standards, which have proven helpful for providing support to survivors of sexualised wartime violence. In spite of this, we know that political, economic and socio-cultural dynamics frequently change the working conditions in our projects. As an organisation, we respond to this by continuous examination of our approach, monitoring its effects and further developing it.
medica mondiale has devised instruments and methods to allow for regular monitoring of the impact of our projects. We also commission external experts to conduct impact monitoring. In this way, we can respond to potential problems and learn from our mistakes. Paying due respect to the cultural specifics of project environments, we assess each regional context and draw up our evaluation system in close cooperation with local staff in partner organisations.
“It is important for us to consider and define impacts
from the perspective of our target groups.
This is also part of our feminist identity.”
Since the beginning of our work in Liberia in 2006, medica mondiale has built a broad support network including expert counsellors in the provincial capitals and volunteer counsellors organised in solidarity groups on community level. They are the first contact for women affected by violence and have all been trained to apply the Stress- and Trauma-sensitive Approach.
To assure quality, facilitate guidance and for continuous improvement, medica mondiale Liberia developed a two-tiered monitoring system of counselling services. It comprises of two main components: knowledge and self-care. It allows staff and volunteer counsellors to check their counselling ability and to determine whether their work complies with best practices and standards. At the same time, the system encourages counsellors to reflect on their work and to ensure self-care.
Stress and trauma-sensitive approach: protection and safety
After all, their own sound health is crucial for successful counselling, especially to offer the all-important feeling of protection and safety. This constitutes the core of the Stress- and Trauma-sensitive Approach.
To ensure usefulness for all counsellors, tools and methods have been designed with a wide range of living and working conditions in mind. For example, the questionnaires contain pictures and symbols instead of text to ensure they are meaningful to women who cannot read and write.
The monitoring of both components – knowledge and self-care – consists of four interdependent steps, one building upon the other:
Step of knowledge, attitudes and skills
Based on this self-reflection, monthly group meetings are held, during which the counsellors share and evaluate each other. Frequently, these meetings also serve to talk about specific cases and to find solutions together.
In a third step and throughout long-term intervals, team leaders or external specialists provide group counselling sessions. Counsellors benefit by receiving feedback helping them to improve their work; the learned is then immediately tested in supervised role-play sessions. It is important to ensure that participants do not have the feeling they are being checked upon. Instead, sessions should be held in a spirit of assistance to further develop their valuable work.
The Monitoring of "self-care" provides staff with practical methods and various options encouraging rest and detachment from their work experience. The first step here involves physical relaxation exercises.
Regular group discussions provide an opportunity to share their experiences with colleagues and experts, to learn how to deal with stress, while also receiving practical instructions on how to support each other.
The third step is team and individual coaching for professional debriefing of the experience.
With the aim of monitoring the quality of trauma-sensitive counselling services, this two-tiered monitoring pyramid has proven very successful. It is more difficult, however, to measure the impact related to "change in attitude" and "changes in society". Has the work of medica mondiale really contributed to improvements in self-confidence amongst the women in the project region? Do they know their rights and actively assert them? Have there been general changes in attitudes towards women or specific gender roles?
“The establishment of 12 Women Support Groups (WSG) made up of women nominated by community members, stimulates community level participation in addressing SGBV and plays a very active role in the prevention and response to SGBV.”
For the project region in Liberia, we can actually answer some of these questions. An evaluation of 2019 showed that thanks to solidarity groups at community level, women and girls now know and perceive their rights and are better protected against sexualised violence. Men were also sensitised to the issue of sexualised violence. As male change agents, they promote non-violent interaction in their communities.
Engaging government institutions in the assessment
In order to combat sexualised violence long term, medica mondiale engages government institutions, e.g. healthcare professionals, officers from the police and/or the justice sector. medica mondiale verifies on a regular basis, if cases of violence are being documented according to the given standards and if trainees apply what they learnt. For example, a 2018 evaluation showed that the number of women who felt empowered by health worker support increased from 67 to 92 per cent over the course of the project in Afghanistan.