The hidden danger of big data

The price of anarchy is a price well worth paying if we want to preserve innovation through serendipity.

by &
    The hidden danger of big data
    In 2015, humankind produced as much information as was created in all previous years of human civilisation, write Ratti and Helbing [Getty Images]

    In game theory, the "price of anarchy" describes how individuals acting in their own self-interest within a larger system tend to reduce that larger system's efficiency. It is a ubiquitous phenomenon, one that almost all of us confront, in some form, on a regular basis.

    For example, if you are a city planner in charge of traffic management, there are two ways you can address traffic flows in your city. Generally, a centralised, top-down approach - one that comprehends the entire system, identifies choke points, and makes changes to eliminate them - will be more efficient than simply letting individual drivers make their own choices on the road, with the assumption that these choices, on aggregate, will lead to an acceptable outcome.

    The first approach reduces the cost of anarchy and makes better use of all available information.

    Inside Story - How safe is your personal information on the internet?

    The price of anarchy

    The world today is awash in data. In 2015, humankind produced as much information as was created in all previous years of human civilisation.

    Every time we send a message, make a call or complete a transaction, we leave digital traces. We are quickly approaching what Italian writer Italo Calvino presciently called the "memory of the world": a full digital copy of our physical universe.

    As the internet expands into new realms of physical space through the internet of things, the price of anarchy will become a crucial metric in our society, and the temptation to eliminate it with the power of big data analytics will grow stronger.

    Examples of this abound. Consider the familiar act of buying a book online through Amazon. Amazon has a mountain of information about all of its users - from their profiles to their search histories to the sentences they highlight in e-books - which it uses to predict what they might want to buy next.

    As in all forms of centralised artificial intelligence, past patterns are used to forecast future ones. Amazon can look at the last 10 books you purchased and, with increasing accuracy, suggest what you might want to read next.

    A certain amount of randomness in our lives allows for new ideas or modes of thinking that would otherwise be missed.


    But here we should consider what is lost when we reduce the level of anarchy. The most meaningful book you should read after those previous 10 is not one that fits neatly into an established pattern, but rather one that surprises or challenges you to look at the world in a different way.

    Contrary to the traffic-flow scenario described above, optimised suggestions - which often amount to a self-fulfilling prophecy of your next purchase - might not be the best paradigm for online book browsing.

    Big data can multiply our options while filtering out things we don't want to see, but there is something to be said for discovering that 11th book through pure serendipity.

    Threat to innovation and democracy

    What is true of book buying is also true for many other systems that are being digitised, such as our cities and societies.

    Centralised municipal systems now use algorithms to monitor urban infrastructure, from traffic lights and subway use to waste disposal and energy delivery.

    Many mayors worldwide are fascinated by the idea of a central control room, such as Rio de Janeiro's IBM-designed operations centre, where city managers can respond to new information in real time.

    OPINION: Internet or splinternet?

    But with centralised algorithms coming to manage every facet of society, data-driven technocracy is threatening to overwhelm innovation and democracy.

    This outcome should be avoided at all costs. Decentralised decision-making is crucial for the enrichment of society.

    Data-driven optimisation, conversely, derives solutions from a predetermined paradigm, which, in its current form, often excludes the transformational or counterintuitive ideas that propel humanity forward.

    A certain amount of randomness in our lives allows for new ideas or modes of thinking that would otherwise be missed.

    And, on a macro scale, it is necessary for life itself. If nature had used predictive algorithms that prevented random mutation in the replication of DNA, our planet would probably still be at the stage of a very optimised single-cell organism.

    Decentralised decision-making can create synergies between human and machine intelligence through processes of natural and artificial co-evolution.

    Distributed intelligence might sometimes reduce efficiency in the short term, but it will ultimately lead to a more creative, diverse and resilient society.

    The price of anarchy is a price well worth paying if we want to preserve innovation through serendipity.

    Carlo Ratti directs the Senseable City Laboratory at the Massachusetts Institute of Technology and heads the World Economic Forum's Global Agenda Council on Future Cities.

    Dirk Helbing is professor of computational social science at the Swiss Federal Institute of Technology (ETH) in Zurich and heads the FuturICT and Nervousnet initiatives.

    The views expressed in this article are the authors' own and do not necessarily reflect Al Jazeera's editorial policy.

    Copyright: Project Syndicate 2016 - The Hidden Danger of Big Data

    SOURCE: Project Syndicate



    Interactive: How does your country vote at the UN?

    Interactive: How does your country vote at the UN?

    We visualised 1.2 million votes at the UN since 1946. What do you think are the biggest issues facing the world today?

    'We were forced out by the government soldiers'

    'We were forced out by the government soldiers'

    We dialled more than 35,000 random phone numbers to paint an accurate picture of displacement across South Sudan.

    Interactive: Plundering Cambodia's forests

    Interactive: Plundering Cambodia's forests

    Meet the man on a mission to take down Cambodia's timber tycoons and expose a rampant illegal cross-border trade.