What kind of data revolution do we need?
“In my book Poor Numbers I have described a basic knowledge problem in economic development statistics. Poor Numbers focused on the provision of data needed for the economic governance of African countries, and concluded that the numbers that we base economic decisions on are currently poor and seriously misleading. To explain this outcome I point to two dominant factors. First, many poor countries currently have weak statistical systems. Recording and data collection is particularly constrained in poor economies. Second, demand for data has been largely external to these poor economies and this demand has been largely uncoordinated and therefore often disruptive to the regular supply of data. As a result statistical offices have become de facto data collection agencies for hire, and the supply of data has been irregular and of very uneven quality. The state of affairs has been called a ‘Statistical Tragedy’ by the World Bank chief economist for Africa.
I suggest that we do not want to repeat past mistakes. On the basis of the diagnosis provided in Poor Numbers I put forward five fundamentals propositions that need to be taken into account before we embark on a data revolution. If we don’t, then this demand for evidence is going to be counterproductive and do more harm than good.
1. We need to focus on data supply, not only on global data demand.
Consistently the development community is repeating the mistake of naively demanding more and better data. The monitoring of specific projects should be tempered by a realistic assessment of the capacity of the statistical office to deliver information. The Millennium Development Goals agenda identified targets, but gave less thought to where the information should come from. We need to turn this important development question upside down. Rather than asking: ‘what kind of development should we target’, the question should be – ‘What kind of development are we able to monitor’?
2. A data revolution has to focus on costs as well as benefits.
More data is only better data if it is meaningful information and there are no opportunity costs to its supply. A useful exercise is to sift through the current 8 goals, 18 indicators and 48 targets and think about the cost of supplying all those data on an annual basis for all poor countries. In practice there are more gaps than real data observations in the global datasheet. Moreover, the supply of one indicator may come at the cost of another. When one member of staff is pulled away from balance of payment statistics in order to provide health statistics, we get more of some data and less of others. If there is a reliable national sampling frame, the costs of a Living Standard Measurement Survey could be up to 2-3 million dollars for an average country and take about 18 months to two years to complete. The supply of data has a cost, and we should rethink the list of indicators with that in mind.
3. Many statistics are public goods, and therefore we need to worry about market failures in their provision.
Not all statistics are public goods. Market research and opinion polls are supplied privately for private demand. But statistics on health, the economy, expenditure and population are public goods. There are well established theorems on why the supply of public goods is a public matter – and therefore mostly, but not always, pertains to the state. There is also an inherent risk of the tragedy of the commons. This occurs when demand overshoots supply, and particularly where private demand only has to cover the private costs (like when per diem is pulling staff out of their office), there is a negative social externality (no staff to complete a task a with lower financial reward) and the market demand for data does not cover the general cost of maintaining a national statistical office.
4. Paying for results is not only naïve but dangerous.
Hand in hand with the demand for a data revolution comes the idea that the development community should not only embrace evidence based policy, but also move towards paying for results. This would be a great idea if poor countries were closely observed labs where you could make objective observations, and the act of observation did not affect the facts. It may be a terrible idea if this is not the case. The data basis is so meager, and the data collection and aggregation process so vulnerable, that if you put forward strong incentives for reaching a particular number the data will be manipulated. This happened to immunization data, poverty rates in Tanzania, primary school rates in Kenya, and maize production in Malawi. What you get is policy driven evidence, the opposite of what we need. If we think that a child is vaccinated, has escaped the poverty line, goes to school and has enough food, when this is not the case, it is not a statistical mistake but a real tragedy
5. We need to worry more about local demand and applicability to have a sustained improvement in data for development.
Local demand for data needs to come into focus. A statistical office is only sustainable if it serves local needs for information. This is the time for the development community to remind ourselves that while we demand evidence for policy, we must not forego the opportunity to invest in accountability. Obtaining data is not a technocratic exercise but rather one of building institutions. For all this talk of ‘institutions matter’ and ‘governance’ in development circles, there has been a surprising gap in analyzing the statistical office. Statistical offices are first and foremost institutions that provide information to promote a discourse between citizens and states. We should spend more resources to find out which data matters for the citizens of these countries.
The statistical offices and their ability to provide timely high quality data on their countries have long been neglected. This is a chance to set these past mistakes right, and to move beyond a statistical tragedy, to stake a path towards statistical capacity building. It might not be a revolution, but considering these fundamentals of provision of data in poor countries, is a step towards better data for development.”
Morten’s post is part of a new blog series on post2015.org on the data revolution. Find out more here.