Big Data, societal challenges and the easy solutions pitfalls

The magnitude and the growth rate of data that we daily produce have increased massively. This data “proliferation” is generated mostly by three sources: machines, companies and humans. According to Digital Universe study, by 2020 around 5.000 of gigabytes will be produced for every individual on the earth, more or less equal to 40 zettabytes. These “numbers” say nothing to most, but put such figures in perspective 40 trillion gigabytes are estimated to be 57 times the amount of all the grains of sand on all the beaches on earth. Mostly (around 90%) of these data are unstructured, meaning that such data does not have a pre-defined structure or pre-defined manner to be managed or analysed. Unstructured data are typically text, sounds, videos, pictures, records, indeed something that we cannot manage using excel sheet for instance. The concept of Big Data emerged as one of the major trends for business leaders, policymakers, IT specialist. Define the concept of Big Data can be quite daunting. In fact, still there is not a widely accepted definition of Big Data. The possibility to discover hidden information or observe an emerging phenomenon is the most challenging component of the Big Data era. These challenges go beyond the “exclusive domains” of IT programmers, ICT scientist, Data scientist, etc., but require the necessity to develop new forms of cooperation and integration of study fields.

The growing data rate within, and of course not only - the European Countries had a significant influence on the recent policy agenda. The European Union decided to massively invest in research and innovation to gain and produce new, sustainable and competitive advantages for the European citizens. Such benefits have the potentiality to create new knowledge, tackle complex problems and propose innovative solutions for the economy and society. Huge benefits are now achievable thanks to the adoption of Big Data technology. For instance, the genome mapping and the application for treating cancers or rare disease. The real-time emissions monitoring to tackle climate changes, new business opportunities for start-ups, and many others. Big Data applications seem to be worthwhile in all sectors. However, apply it in concrete applications for practical operations is far to be an easy task.

Figure 1 show, how interconnected is the world in which we live. The figure has been created downloading from the European Commission official Facebook page, the connections across the users’ employing the network analysis technique. Each colour is a topic of discussion (i.e. health, economy, policy, law, etc), and the size of the labels are proportioned to the likes (a sort of attentions) received by the user.

Even if Big Data and all the newest ICT techniques allows to explore, analyze and graph complex phenomenon, we must need to keep in mind that the complexity is massive, and the number of variables which interact each-other even more. The complexity behind global issues, such as the economic and financial crisis, the migration drama, the climate change, or the food emergency is huge. However, despite the huge amount of data, and potentially knowledge, available like never before in human history, we are approaching these common issues convinced to solve them, as individuals, acting as “little God” in our expertise domain. We are acting in embracing the God complex, which is a refusal to admit the possibility of being wrong regardless of the complexity of the situation. As the figure shows, Big Data, right now, underling the complexity and the intricate nature of the world rather than offer an “easy” tool to solve them. So what do we do now? How do we solve complex problems? No definitive answer exists!! Nonetheless, as Tim Harford suggested in his Ted Talk: “first begins with humility and acceptance that we might not have all the answers.” Then, the next step involves applying common sense to figure out what works and what doesn’t. The take-home message is to adapt quickly, experiment often, and continually tries new things.

Economics writer Tim Harford studies complex systems and finds a surprising link among the successful ones: they were built through trial and error. In this sparkling talk from TEDGlobal 2011, he asks us to embrace our randomness and start making better mistakes. This talk was presented at an official TED conference, and was featured by our editors on the home page.