BLOG - Open data; no value without Big Data

However, more data is not necessarily better. In fact, some consumers get stressed out by choice overload if they have too many options: too many types of eggs or brands of coffee, where it seems initially that the only difference is in the information about the production methods. Sometimes economists prefer less data from a random sample, provided the quality is good, to poor-quality data from hundreds of businesses.

More data is not always better, because economists are more aware than anyone that data is not free, even though ICT developments are rapidly lowering the costs of collecting and disseminating it. Countless devices - anything from milking robots to spray machines - are churning out data thick and fast; the ‘Internet of Things’ concept, which, in turn, leads to the analysis of big data.

Now that it’s so much cheaper to make data available, it has been suggested - in line with the Open Data idea - that government agencies open their datasets free of charge. After all, we - the taxpayers - have already paid for that data, so it belongs to all of us. Developers of smartphone apps will then get free access to material and be able to provide us all with services.

If all accounts are true, great benefits can be reaped from open data. EU Commissioner Neelie Kroes has even spoken of an ‘economic bonanza’. In a recent report by McKinsey & Co. extra added value of between 3 and 5 billion dollars was predicted, an astronomical sum that I can barely imagine. Then there are the environmental spin-offs and greater transparency in government.

At LEI, we are involved in a few projects involving open data. And we also foresee huge benefits. Wouldn’t it be great if, as a fruit grower or trader, you knew all the ships that sailed laden with apples from the ports of New Zealand and Chile and exactly when they were due to arrive in Rotterdam and enter the European market? That kind of information could prove very useful in market intelligence.

But that raises all sorts of other questions in the process. Would the importer of that fruit like that idea or would his interests actually be better served by less transparency in the market? Who covers the costs incurred by the government for making the data available in the first place? Wouldn’t that lead to false competition with market players who currently offer commercial information in expensive reports – or do they stand to benefit as well?

Economists point out that you can only realise value from a product or service if you own it. If you have no business model to assure you of a return on your app, that free open data will never be converted into a new service – unless people write open source software in their spare time or with grant-funding.

One intriguing aspect of open data in the agricultural sector is that it goes totally against the trend: traditionally, open data has always been available in our sector: from LEI price statistics to the harvest estimates and auction data of Statistics Netherlands. But these sources are disappearing. Products are becoming more heterogeneous, more products are being made to order and there is less trading in open markets. The government is taking more of a back seat and leaving market information to the chains or service providers, such as the agricultural media or consultancies. Product boards are being scrapped and some farmers are having to set up their own information-exchange groups – which could lead to friction with the trade or even breach the competition legislation.

Open data might reverse this trend, but it will only really come into its own when it can be linked to the big data in the computers of the businesses communities in the agricultural chains, including retail.