Monitoring, Feedback and Evaluation: three new keys to organizational learning
In an age when organizations are increasingly keen to arrive at evidence-based decisions, managers are faced by questions such as: What does this dataset tell me? Which data will actually help me to come to a decision?
Although the capacity to gather and analyze information has grown exponentially in the digital age, drawing conclusions and coming to decisions based on these analyses is still an arduous task, both for those who work with data and for decision-makers themselves.
Eager for data, organizations introduce protocols and tools to be applied by collaborators to their clients/beneficiaries. This ends up creating bureaucratic structures that take up time that could be spent on the actual work of the collaborators. Furthermore, these tools and protocols generate such a vast quantity of sometimes meaningless or redundant data that the organization may not even have the capacity to handle it.
The field of monitoring and evaluation is directly affected by this: while it is subject to heavy demand from “data-driven organizations”, it ends up producing reports that do not meet the concerns of decision-makers and are often hard to understand even by those who requested them.
These contemporary management needs have given rise to a movement that aims to operate precisely in this gap between managers and the information generated by more traditional fields of management research. This movement is called Feedback Loop. While monitoring delivers information on outputs and thus the progress of implementation of actions, and evaluation, in turn, analyses the outcomes or the efficacy and efficiency of the process, the feedback loop aims to provide something that is as important as it is obvious: a connection between managers and the real world in which their work takes place.
To achieve this, the feedback loop undertakes simple research and is less concerned with the methodological rigor typical of traditional evaluations. It aims to furnish more digestible information within a short period of time. This has helped organizations, for example, to gain a clear idea of what kind of actions people in a certain area expect from a social project or of what they think of the actions that are being undertaken. Likewise, it can provide information on the opinions of stakeholders regarding their work with their funding agencies or on the immediately felt effects of a project.
Using questionnaires with no more than 10 quantitative and qualitative questions often organized around a Net Promote Score, or using well-guided systematic focus groups, the feedback loop has been able to help managers to find their way across the ocean of information with which they are nowadays inundated.
It is important to acknowledge the limitations of a methodology that does not collect sufficient information to fully understand a phenomenon and does not do extensive work on ensuring that samples are representative of the target population. However, these shortcomings have not proved to be a barrier to advocates of this new way of providing evidence for managers. Many report that they find it difficult to make investment decisions based solely on data relating to the progress of actions, without information on the real situation on the ground, or using evaluations that often take months to prepare and see the situation only from the perspective of the expected outcomes.
Feedback connects people. Within a very short space of time, for example, a health service can have a clear idea of what patients think of their doctor’s appointments and what they would most like health professionals to change. A funding agency can gather the opinions of all those involved in a project financed by it and help stakeholders to adjust their strategies accordingly. A group of professionals who work in a private social investment firm can reach out to those who benefit from their work, understand their real needs, change direction and even review their practices so as to better address the concerns of the target population.
The creation of a system of organizational learning based on these three pillars has been highly successful in North American and European philanthropic organizations. The combination of these three types of data collection and analysis have also helped to reshape their work entirely, cutting out waste and improving efficiency.
By Gustavo Valentim
Daniel Brandão and Gustavo Valentim were in the US for meetings at FSG, one of the largest global consultancy organizations in the field of social impact, and at MIT’s D-Lab, for conversations with Acumen and to participate in Feedback Labs, a conference attuned to the latest world trends in lean management, in which feedback is a key strategic tool.