The Big Data revolution started 2 centuries ago

Posted By Pierre-Nicolas Schwab on 27 Apr, 2018

In her latest article Antoinette Rouvroy develops an interesting thesis
The current Big Data revolution would be the continuation of earlier shifts. Let’s see in more details.

Numbers and statistics increasingly being used since 19th century

Big Data is based on the massive collection of data and would be, according to Rouvroy, an extension of earlier shifts that started with the increasing use of numbers in the eighteenth century, and the emergence of statistics in the nineteenth century. A good illustration of the latter is one of the first “Big Data” visualisations ever by Charles Booth (see our article here).
Rouvroy cites Hacking who speaks of an “avalanche of printed numbers” during the nineteenth century. Here’s an exerpt of the book by Hacking :

“Graunt and the English began the public use of statistics. Peoples of the Italian peninsula and elsewhere had promulgated the modern notion of the state. But it was German thinkers and statesmen who brought to full consciousness the idea that the nation-state is essentially charaererized by its statistics, and therefore demands a statistical office in order to define itself and its power.”

Beyond the visible (and the understandable) : looking for unknown correlations

All algorithms are based on the detection of patterns, the calculation of correlations between seemingly unrelated variables. The user whose data is processed becomes a “model” in itself.
Rouvroy proposes that correlation and prediction based on “small patterns” follow a trend set up by late nineteenth criminal investigation methods which used small evidences to confound criminals or, in the Arts, the Morelli method which “is based on clues offered by trifling details rather than identities of composition and subject matter or other broad treatments that are more likely to be seized upon by students, copyists and imitators.” (wikipedia)

What distinguishes the Big Data revolution from earlier shifts ?

Rouvroy sees the differences of the current Big Data revolution with the earlier shifts in the following aspects :

• Algorithms make the link to the “sensitive” world disappear : in other words data is captured out of context
• Algorithms detect patterns rather than what we’ve done, who we are, why we do things
• Algorithms allow the transition from reaction to anticipation.