A monitoring system provides continuous real-time information about how your program is being implemented and how you’re progressing toward your goals. It helps you identify which parts of implementation are going well and which are not.
You can use monitoring data to manage performance and learn where your program is not working as expected. For example, if you are running a teacher training program, you could track which teachers did not attend the training and whether teachers who attended implemented what they learned. You can then quickly identify which goals are not being met, and you would know how to investigate why. For example, if you find that only a small percentage of teachers are using the material, you can conduct follow-up interviews to understand the underlying reasons. This information can then inform improvements to your program.
Which questions can a needs assessment answer?
Tracking implementation will help you identify which areas of your program should be strengthened. For example, consider a program that provides information to farmers through phone calls that play pre-recorded material. Tracking whether people are answering the calls and listening to the full message can help you identify problems early on. You may learn that your phone numbers are out of date or that the numbers do not belong to your target audience (e.g. you may be reaching migrant workers in cities rather than rural farmers).
Generating evidence for decision-making is just one use of monitoring –a monitoring system is often used for other functions such as time reporting, payroll, and staffing. Donors, in particular, may expect monitoring data for accountability, to ensure you are making good use of program funds.
Next steps - Conducting monitoring
Monitoring requires that your organization collect, store, analyze, and visualize data on an ongoing basis. The data should be presented in a format that is useful for decision-makers. Using this data should be an established part of organizational decision-making.
Should you set up a monitoring system yourself or hire an expert?
Ultimately, your organization will need to conduct monitoring on its own – so your system needs to be set up in a way that program staff can use it independently. However, setting up the monitoring system can be done in-house or be outsourced.
Monitoring should be closely integrated with implementation. Since you are the best expert on what your program is trying to achieve and how it works, you should plan to be closely involved in setting up the monitoring system. You may want to hire an external expert if you do not have the technical capacity to set up a system for collecting, storing and presenting data in useful ways. It can also be useful to get an expert’s feedback on ways to measure the indicators you need.
How to conduct monitoring
Our step-by-step instructions and template will help you develop a monitoring system suited to your program needs.
Use the Theory of Change Builder to create a clear narrative for why your program will work.
Already created a theory of change? Log in and make sure it is up to date! We’ll refer to it going forward.
You can still proceed without linking your theory of change, if you want to preview the steps or have a theory of change saved elsewhere.
An implementation plan is more detailed than a theory of change, and includes the details of how you’ll put your program into effect – who will do what with which resources and materials.
An implementation plan provides the operational details of how a program will run – who will do what with which resources. It also includes when, where, and how frequently these activities will happen. Your implementation plan should also include benchmarks for program success, such as targets for program uptake, service delivery, or program participant satisfaction.
When an implementation plan is not written down in detail, there may be a lot of implicit (and potentially conflicting) knowledge about what should be happening. It is important to document program details so you can compare what’s happening on the ground to what was expected.
These may be shaky assumptions that you want to test, potential performance issues you should check for, or expected results that are closely tied to your impact. For example, say you are running a program in which health workers go door-to-door to remind parents to take their children to local vaccination camps. Some steps you may want to monitor are:
- Do health workers receive timely information on when vaccination camps will be held in the health center?
- Are health workers reaching their weekly targets for number of home visits?
You can identify shaky steps by asking:
1. How critical is this step for program success?
2. How certain are we that this assumption will hold in practice?
The Theory of Change Builder automates this process for you! Log in to access a saved theory of change, and we’ll guide you towards identifying shaky steps. If you’re working with an offline theory of change, you can follow these instructions to identify shaky steps.
The indicators you choose should meaningfully answer your questions and be easy to measure reliably. Most of the time, you can identify indicators already in use rather than inventing your own.
For example, here are some sample indicators for the health worker home visit program:
While the Impact Measurement Guide focuses on data to measure and improve program performance, your monitoring system can also help you collect data for other purposes, such as payroll, staffing and donor reporting requirements. For example, some donors may ask for demographic information about people participating in the program.
Create a response framework that describes whether and how your organization will act on the data you collect. Often, this means setting thresholds for indicators, and defining meaningful actions your organization will take if these thresholds are crossed. As an example, have a look at this sample response framework for the three indicators we created for the heath worker home visit program.
A good rule of thumb is: if you can’t think of an action your data would inform, you may not need that indicator!
Most nonprofits will use administrative data that is already available to them, either collected on their own or by government agencies. The data collected at registration or data measuring program activity can prove to be valuable for the purpose of monitoring and observing whether program targets are being met.
To identify which data collection method would suit your needs, you need to know: Who or what do you need data on? What is the best way to obtain it? How often does data need to be collected?
In some cases where new data needs to be collected for monitoring, surveys can be conducted. For example, to gauge the level of satisfaction of the parents bringing their children to vaccination centres, satisfaction surveys can be conducted while vaccinations centres are running.
You also need to consider how to record the data you are collecting. Typically, organizations conduct surveys in-person or over the phone. You can set up a simple data collection system that use pen-and-paper surveys, direct data entry into spreadsheets, or digital surveys. Click here for more guidance on developing a data collection system.
Where will the data you collect go? How will it be stored? For basic monitoring systems, data can be stored as data files (.xlsx, .csv etc.) in the cloud using Dropbox or Google Drive. This ensures that data can easily be accessed using commonly-used spreadsheet software (such as Microsoft Excel or Google Sheets) – and it also backs up your data. As you set up these technologies, also remember that you also need to train staff to maintain and operate these systems.
As your organizations grows and your needs become more complex, you may decide to invest in more expensive and/or customized systems, such as an SQL database server.
It’s important to develop strong data security practices that protect the identity and privacy of your respondents, particularly if you are collecting confidential or sensitive information. Paper survey responses should be stored securely (e.g. in a locked cabinet), and electronic data should be protected with strong passwords. You can also consider encrypting your data.
Preparing the data for analysis is called data cleaning. A large part of this process involves addressing data entry errors. This should be automated as much as possible in the previous steps. For example, if you are asking respondents their phone number in a digital survey, you can set a constraint that the response be exactly ten digits long.
A monitoring system is useful only if it is regularly informing organizational decision-making – so make sure your response framework is integrated into the monitoring system! Does the information from your monitoring system reach the right decision-makers? Does this information arrive in time and in a format they can use easily? Does this information translate into meaningful actions to improve the program?
Congratulations! If you’ve come this far, you’ve set up a monitoring system that is giving you regular, actionable information about how your program is performing. Now, your challenge is to ensure your monitoring system continues to be responsive to your program’s needs. As you resolve certain problems and shift your focus to others, you will need to revise the indicators your monitoring system collects. Periodically review whether your monitoring system is giving you all the answers you need, and whether you are using all the data your monitoring system is collecting.
You can use this checklist to assess your monitoring system and identify areas of improvement.