A Framework for Digitising Monitoring for Better Development Outcomes

Effective monitoring of government policy and program implementation improves public expenditure efficiency. Monitoring can generate real-time data across the results chain—resources, activities, outputs, outcomes, and impact—serving several public goods. Traditional paper-based monitoring has been less effective because of poor quality of data, time lag, limited analysis and insights, etc. Digitisation of program and policy implementation has improved the quality of monitoring and effectiveness of public expenditure. Yet, digitisation has not delivered its promised full potential. Often, it is because of a weakness in tracking outputs and outcomes. Digitisation of monitoring varies across different departments and levels of government. This paper discusses a framework to help deepen the digitisation of monitoring, especially at the state and local body level, based on NITI Aayog’s implementation of the Data Governance Quality Index (DGQI) in the central government. 

A data value chain for monitoring comprises data collection, cleaning and processing, quality assurance, storage, integration, analysis and visualisation, interpretation, dissemination, and archiving. The full benefits of digitising monitoring involve reviewing and strengthening the whole data value chain. A comprehensive framework approach encompassing the data value chain is critical to tapping the digitisation process’s full potential to enhance monitoring of government policies and programs. 

The framework can help agencies review and improve digitisation across the data value chain for monitoring. With complementary toolkits developed by NITI Aayog, the framework can assist agencies in self-assessing the digitisation process and developing roadmaps for digital monitoring transformation. 

A framework for digitising monitoring

The framework comprises three broad categories. Data system, the first category, handles data collection and interpretation in the value chain. The second category is data outcomes deals with data generation to track development outcomes. Institutional arrangements needed for driving the monitoring digitisation process is the third category.

Data systems

The data system includes data generation, quality assurance, data analysis, use of technology, data security, and data management protocols. Data generation deals with the level and sophistication of data generation. The gold standard for data generation is real-time transactional data with limited or no human interphase in data generation. Deploying relevant sensors generates additional data, such as geolocation. The sophistication of data generation includes the granularity, frequency, use of computer-aided personal interview (CAPI) tools, geographical information systems, transactional data, and level where data digitisation takes place when the original data is collected on paper. 

Quality assurance covers data cleaning, addressing missing data, systems to evaluate data quality, back checks and other mechanisms to validate data quality. Data analysis should maximise use and lead to meaningful insights. Various statistical tools and IT software help automate data analysis to provide patterns, trends, and correlations to assist decision-making. Building capacity and incentives for frontline workers involved in data entry and collection is equally important. Data visualisation makes it easier to interpret data by highlighting key trends. Graphs, charts, and dashboards assist in data visualisation. Data interpretation and visualisation lead to actionable insights that help in informed decision-making. 

Technology use leads to efficiencies across the data value chain. This includes the use of Aadhar and mobile phone linkages; linkages to related platforms through API, including PFMS, GSTN, and Udhyog Aadhar; use of local government directory to standardise names of towns and villages; etc. Technology should be deepened to tap into big data from social media, remote sensing, night light, mobility and other non-conventional sources. Better use of the Internet of Things, sensors, GIS, etc., needs to be explored. Data security and privacy involve access controls, regular security audits, masking and anonymisation, encryption, and security awareness training for the staff. 

Data-driven outcomes

Outputs result from activities an agency undertakes over which the agency has significant control. Outcomes and impact result from outputs on individual beneficiaries, communities, and the economy. Failure to translate outputs into outcomes is the main reason for public expenditure inefficiencies. Tracking real-time outcomes will help with mid-course correction and improve development outcomes. 

Collecting outcomes and impact data compared to outputs is a lot more challenging. For example, increased child immunisation (outputs) reduces communicable disease incidence (outcomes). Over a more extended period, immunisation leads to healthier children and better learning outcomes, which are impacts. Both outcomes and impacts are primarily outside the realm of control of the agency, mainly in individuals, households or communities, and therefore, special efforts are required to track outcomes and impacts, often through special surveys. The agency can infer the output data from the administrative data it collects with immunisation. The outcomes data on reducing childhood communicable diseases will need household survey data. 

Digitisation has the potential to help track outcomes and impacts better if agencies’ digital readiness and preparedness to triangulate multiple datasets is optimised. Agencies can track outcomes through a primary survey of beneficiaries. By leveraging technology, agencies can enhance primary surveys’ cost efficiency, quality, and timeliness. Agencies can explore other data sets generated within the agency or by different government agencies that can provide insights into the progress of related proxy outcomes. However, this will require open data protocols for sharing data across departments. Private sector-generated data sets can also help track outcomes. Data generated by social media and non-conventional sources like mobility, nightlight, and other big data can also track outcomes. Data philanthropy and regulations are required to tap private sector-generated data for the public good. Data from monitoring and triangulating with outcomes data from surveys and other sources makes it possible to track outcomes over shorter time frames. 

Institutional arrangements

Digital transformation for better monitoring requires institutional transformation. The foremost organisational change is to remove data silos within an agency. Data silos in a department result from legacy based on project and program-level monitoring. Breaking silos is very challenging. It can be done only with a firm commitment from the top.

A data and strategy unit reporting directly to the CEO or Secretary will be critical to break the data silos and building a culture of intra-agency data sharing. The role of the data and strategy unit is to link all the data generated in a department or agency and provide actionable insights through better data analytics. The role of the data and strategy unit should also include figuring out measures to track outcomes using proxy data generated by different sources, both public and private and conventional and non-conventional sources. 

Making the best use of data through data analytics will need statistics, data sciences, econometrics, and technology capabilities. A department or agency should consider different options to build the human resource capabilities of the data and strategy unit. Given the rapid pace of technological changes, it is essential to implement a human resource strategy that builds on the external ecosystem’s capabilities by hiring consultants and building strategic knowledge partnerships with think tanks, universities, and other relevant non-profits. 

Data Governance Quality Index

The Development Monitoring and Evaluation Organisation (DMEO), NITI Aayog, developed the Data Governance and Quality Index (DGQI). The various ministries and departments of the government of India utilised DGQI to assess the digitisation of schemes and monitor programs. Toolkit development by DMEO helped ministries of the Government of India undertake self-assessment and initiatives to deepen the digitisation of monitoring of various schemes and policies. Data and strategy units were established to develop roadmaps for the digital transformation of the monitoring systems. The action plans for the road maps and the digital transformation progress were tracked. DGQI, as per a report released in 2023, helped many departments of the government of India make significant progress in the digital transformation for better monitoring and delivering development results. State governments and local bodies can undertake similar efforts to improve monitoring and deliver better outcomes. 

Conclusion

A key reason for the suboptimal impact of public expenditure is the failure of public agencies to translate resources, activities, and output into development outcomes. Regular tracking of outputs and outcomes of policies and programs helps to make midcourse corrections to achieve development outcomes more effectively. Regularly tracking development outcomes can pose challenges, but organisations can overcome them by digitising monitoring. The government departments and their agencies must bolster digitisation across the whole data value chain for monitoring. The framework and the toolkit developed by NITI Aayog can help state governments and local bodies digitally transform their monitoring systems to achieve better development outcomes.