Standard 11: Evidence based, action-oriented project management.
Make collaborative, timely, and informed decisions to ensure that project activities deliver intended impact to participants within the approved time, scope, and budget.
Collect and analyze qualitative and quantitative monitoring data and community feedback relevant to project decision-making and learning needs.
-
Why
Collecting and analyzing project monitoring and feedback data supports adaptive managementAdaptive management is an approach to tackling complex challenges. The starting point is an assumption of uncertainty about what will work to address the challenge. It is then characterized by a flexible approach involving testing, monitoring, getting feedback and – crucially – making course-corrections if necessary. (Adaptive management: What it means for civil society organisations, Bond, 2016) and improves project outcomes. Specifically, project managers need accurate, relevant and timely information to:
- Assess project progress.
- Inform ongoing problem solving and good management decisions.
- Understand the perspective of participants and their satisfaction with the project.
- Address feedback and issues raised by community members, both participants and non-participants.
- Contribute to CRS, partner, and wider stakeholder learning.
- Report to project stakeholders including donors.
-
Who
- Primary responsible: Project MEAL coordinator (i.e. MEAL officer) and project manager (PM) or chief of party (CoP)
- The project MEAL coordinator is responsible for developing and field-testing data collection tools, training data collectors, conducting data-quality checks, and analyzing the monitoring data and feedback received per the MEAL plan. The PM/CoP ensures the collection and analysis of monitoring data and community feedback per the roles and responsibilities outlined in the project MEAL plan and detailed implementation plan (DIP).
- Others involved: Partner MEAL and programming staff; project sector leads; other CRS project team members (program and operations, especially any sector leads)
- Partner staff often collect monitoring data, receive and respond to community feedback, participate in data quality checks and contribute to the analysis and interpretation of results;
- Other CRS project team members, particularly any sector leads, participate in data collection and analysis.
-
When
- Ongoing throughout project implementation:
- Regular collection of monitoring data per MEAL plan and DIP
- Regular checks on data quality
- Light monitoring to check for early signs of progress and to see if assumptions are holding true
- Ongoing monitoring of changes in context and project assumptions to improve project decision making and planning (MEAL good practice 2.C).
- OngoingAt least monthly, as per MEAL plan. analysis of monitoring data and of feedback received
- Regular (at least quarterly) in-depth data analysis as preparation for project review and planning meetings (see Standard 11, key action 4 for guidance on meetings).
-
How
Follow these steps to ensure the collection and analysis of quality monitoring data and feedback:
- The project MEAL coordinator ensures project staff regularly collect quality monitoring data, following these good practices:
- Field test all monitoring toolsEach monitoring tool should include clear instructions for data collection (develop tools and instructions following SMILER+ guidance). prior to data collection.
- Train data collectors on quality data collection practices, including obtaining consentThis should include asking for and obtaining participants' consent to collection of their personally identifiable information (PII), as well as informing participants about what will happen to their PII, including with whom it will be shared and why, and when it will be destroyed. See the Informed Consent documents on the MyCRS Responsible Data page. for collection of project participants' personally identifiable information (PII) as applicable, and secure and confidential handling, sharing, and storage of project participants' PII.
- Provide an opportunity for data collectors to practice use of tools prior to data collection.
- Minimize the collection of personally identifiable information during monitoring to avoid data protection and privacy risks to respondents (MEAL good practice 2.A).
- Use a sampling strategy appropriate to project indicators and MEAL plans.
- Collect monitoring data according to the project MEAL plan and related content in the DIP.
- Use field visitMPP 2.1: Conduct quarterly field visits to observe anticipated and unanticipated changes, communicate with community members, and conduct routine data quality assurance checks and other opportunities to informally monitor changes in the project context and critical assumptions; discuss these changes with the wider project team (see Standard 11, key action 2) and update the project risk register and/or issues log accordingly.
- Monitor participant satisfaction with project interventions as part of ongoing data collection.
- See CRS Guidance on Monitoring and Evaluation for resources on field testing, data collector training, and sampling.
- The PM/CoP manages the functioning of project feedback mechanismsMPP 6.1: Establish feedback-and-response mechanisms that are accessible to all community members and in line with theCRS Protection Policy.
MPP 6.2: Respond to community feedback in a timely and safe manner. and ensures project responsiveness to feedback received.
- Respond to project feedback promptly; respond to sensitive feedback according to CRS policy on safeguarding where applicable.
- Document all feedback received and responses given (be sure to document dates).
- Check accessibility and effectiveness of feedback and response mechanismsSee also MEAL Good Practice 3.A: Check on the effectiveness of feedback-and-response mechanisms during evaluations to ensure they are accessible to all community members and remain supportive of the Protection Policy. during monitoring and evaluation efforts (refer to the FCRM Tool #12: FCRM Effectiveness Check).
- The project MEAL coordinator, with support from sector staff and other MEAL colleagues as needed, analyzes monitoring data and community feedback to inform project decision-making.
- Review monitoring data and feedback on an ongoing basis to identify problems or challenges arising with the project, including trends that may be going in the wrong direction. See Monitoring for Problem Solving, Adaptive Management, Reporting and Learning for additional guidance.
- Begin analysis by triangulating data, that is, organizing the data by the type of respondent or group of respondents (e.g. female FGD) to identify common themes (or “codes”) that emerge from that perspective within and across data collection methods. As you identify themes, be sure to reflect on data outliers and less common responses as these may yield insights into iterative analysis plans, new learning questions or additional data collection needs. Then compare themes across different perspectives (e.g. male vs female FGD) to see what is the same and what is different among types of respondents.
- Collect additional data as needed to probe possible issues or problems, including those identified via feedback mechanisms and related to participant satisfaction. Pay special attention to recurring complaints or issues from the feedback mechanisms that may merit immediate discussion and possible action.
- Conduct light checks“Light checks” are essentially spot-checking exercises, using a limited number of randomly selected forms from enumerators, partner organizations, etc. Light checks on data quality include regular review of data completeness (e.g., that forms contain all necessary information) and, if a database is used, comparison of data from original forms with what is entered into the database. Light checks help to identify and address any potential data issues immediately, thereby minimizing errors and ensure data quality. of data quality on an ongoing basis (see MEAL Good Practice 2.BMEAL Good Practice 2.B: Conduct ongoing quality assurance checks of activity records to ensure quality and minimize data risk. ).
- Calculate indicators (as appropriate) and populate the Indicator Performance Tracking Table (IPTT) or project equivalent (e.g. Indicator Tracking Table, quarterly reporting table), flagging for deeper project team review any results that are significantly off-track.
- Disaggregate monitoring data and feedback by sex and other key vulnerability characteristics as defined in the MEAL plan or equivalent. (MEAL good practice 2.D)
- See Standard 11, key actions 2 and 4 for guidance on using this information in project decision-making.
-
Partnership
- In many cases, partner staff collect monitoring data per the MEAL plan and work directly with feedback mechanisms. Some partners may take the lead in establishing the feedback mechanisms, orienting communities to the feedback and response process, and responding directly to feedback received.
- CRS engages partners in quarterly reflectionsMPP 2.4: Reflect on a quarterly basis with partners on monitoring data and community feedback, to inform ongoing and adaptive decision-making and action planning on monitoring data and community feedback. See Standard 11, key action 4 for detailed guidance.
When CRS is a sub-recipient- Follow the same process, incorporating additional requirements or directives from the prime.
Emergency projects- In emergencies, monitoring follows the principle of “Count, Check, Change, Communicate”. We count the good or services delivered for accountability and reporting purposes; we check the relevance, usefulness and satisfaction with the response through light,“Light” data collection in an emergency response refers to collecting data from a small (typically random, but sometimes purposeful) sample of individuals served by the emergency response. time-bound qualitative or quantitative data collection; we make necessary changes in a timely manner to address identified issues and opportunities; and we communicate results, challenges and changes proposed to relevant stakeholders (including donors).
- Adapt the frequency of monitoring activities to reflect the fast-changing pace of the emergency and the response. In the early phases of an emergency response, monitoring data is often analyzed daily as part of participatory “daily debriefs”.
- In emergencies, there is also a stronger emphasis on informal monitoring of changes in contexts and community needs.
- Document and analyze information learned through informal monitoring during daily debriefs.
- Address feedback immediately, i.e. generally within one week.See MEAL Procedure 6.2: Respond to community feedback in a timely and safe manner.
- If you are working on a large-scale emergency where several Caritas Internationalis Members are responding and/or supporting a response of the national Caritas, please refer to the Caritas Internationalis Protocol for CI Coordination in Emergency Response, Emergency Framework and Toolkit for Emergency Response documents. Documents are available in English, French, and Spanish and include guidance on coordination and the process of monitoring and reporting on an Emergency Appeal for funding via the CI Network. If you are not registered on the CI Baobab site, please register here.When registering for the CI Baobab site, CRS staff should select "Caritas United States - CRS" as their organization and list the Humanitarian Response Department and [email protected] as the reference contact. If you have any questions, please contact CRS’ Humanitarian Response Department ([email protected]).
Key resources
Tools and templates
-
FCRM Effectiveness Check (FCRM Tool #12)
-
Feedback and Response Registry Template (see MEAL Procedure 6.2)
-
Feedback Mechanisms Documentation (from MEAL Procedure 6.1)
Policies and procedures
Other resources
-
Institutional Strengthening: Building Strong Management Processes (ISG): Chapter 10, Monitoring and Evaluation
-
MEAL in Emergencies online course (CRS Learns), specifically building blocks on Monitoring Methods; Accountability; Analysis, Interpretation and Use
- Ongoing throughout project implementation:
- Primary responsible: Project MEAL coordinator (i.e. MEAL officer) and project manager (PM) or chief of party (CoP)