The computer abolishes the human past by making it entirely present. It makes natural and necessary a dialogue among cultures which is as intimate, as private as speech.
McLuhan -1968 - War & Peace in the Global Village p.90
I'm exploring a line of thinking about the implications of Big Data - ubiquitous big data. Bear with me, because I will begin and end with McLuhan's quote.
Like all of my blog posts this is a rambling assemblage of ideas - I apologize for any incoherence and all the errors are mine. However, the articles/books/videos/blogs I point to are great.
I'm not a physicist - but this is what I've understood from reading Manuel DeLanda (who is well worth reading and viewing - he's got lots of material that you can Google and available for free).
The digital environment is an intense medium - a medium determined by dimension of pressure, temperature, density, and connectivity (there are more types of intensities - but these are reasonable to think about). Unlike 'extensive' dimensions like width, length, etc. intensive dimensions are subject to certain types of change.
For example temperature of water - from 99 Celsius to 1 Celsius (a change of 98 degrees) temperature seems to impart a linear change on the water - slowing feeling colder. But from 1 Celsius to -1 Celsius (a change of 2 degrees) something profound happens. Water becomes a solid - the difference in temperature is not the really important thing to discern - it's that the very nature of what it is seems to have been transformed.
Another example can be seen when we look at changes in population density. In my last post I spoke of the increase in population density as humans shifted from hunter-gathering to agriculture and then to industrial societies which enabled associated increases in divisions-of-labour and related modes of production. But it is more than just an exponential increase in divisions of labour that characterizes these phase transitions. Each transition includes a corresponding increase in new types of institutions and new social structures, social forms and of course new ways of being. I highly recommend this book.
The idea of a 'phase transition' arising out of increases of population density (and other social intensities) was an important insight for me.
I haven't finished his book yet, but I'm reading Dougla Allen's book,
The book blurb at Amazon says this: In The Institutional Revolution, Douglas W. Allen offers a thought-provoking account of another, quieter revolution that took place at the end of the eighteenth century and allowed for the full exploitation of the many new technological innovations. Fundamental to this shift were dramatic changes in institutions, or the rules that govern society, which reflected significant improvements in the ability to measure performance—whether of government officials, laborers, or naval officers—thereby reducing the role of nature and the hazards of variance in daily affairs. Along the way, Allen provides readers with a fascinating explanation of the critical roles played by seemingly bizarre institutions, from dueling to the purchase of one’s rank in the British Army.
I highly recommend this book. It provides a powerful framework for understand the disruptive impact of Big Data for evolving our current institution. Ubiquitous access to Big Data will enable (in the sense that Stuart Kaufman suggests) whole new forms of measure - measures of things we have thought inconceivable to measure before. And these measures will not be simple aggregates - averages but they will make visible the dynamic turbulences of social behavior and even of consciousness. This in turn will enable unforeseeable institutions, including those involved with work and organizations.
Two great Edge Conversations are must reads for understanding Big Data.
In this conversation Sandy Pentland does a wonderful job elaborating new forms of measure that Big Data can enable.
The next Edge Conversation that is a must read/listen is Cesar Hidalgo's
He provides the best definition of Big Data that I've read so far. Big Data is even more than when more becomes different. He suggests that there is at least three dimensions to Big Data:
1. Size: It has to be three times big. It’s not about 10, 100 or a 1,000 people. It has to be about millions of people or millions of entities.
2. Resolution: It has to be big in resolution. It’s not just knowing an average or an aggregate over a long period of time, or over a large amount of space. It has to have a capability to have a high spatial resolution, high temporal resolution, and high typological resolution. For example, in looking at the spending patterns of people, it’s not about their average income. That's very aggregate. It’s about where they spend, at what time, and what things they purchase together. That would make data big.
3. Scope: It has to be big on scope. Many confuse ‘lots of data’ with Big Data. It need to be useful for understanding things other than your core business, to understand things about the world. That's a condition that is very hard to get. There can lots of data about the procurement process or the value chain of shoelaces. It can have very fine spatial and temporal resolution. There might be lots of shoelaces that are crossing boundaries, so the data might be big in that dimension, but you're not going to understand anything other than the shoelace industry from that, so the data would not be big in scope.
The key insight I got from Cesar's conversation is when he discusses the difference between money and value. He frames it as the difference between wealth and value - which I think is not quite right because the way he talks about value seem to me to be identical to the way Adam Smith talks about wealth (but that is another discussion). The issues is the difference between the value we generate and the value that can be 'appropriated' for what is generated. For a long time I've wondered where Wikipedia shows up in our GDP measures. The value generated by all the contributors, by all the users who are able to use the value they find on Wikipedia to add value to their own efforts to understand or answer questions. Unlike Facebook, Wikipedia is a non-profit foundation and therefore is much less concerned with the need to monetize the value they generate.
Why this is important is that money is now simply information - bits transferred via digital means and measured via digital means. It is a subjective perception of value that is measure through the 'illusion of precision' of numbers. Like Olympic judges with their score - e.g. 7.8 via 9.3 - we don't know if the 7.8 is actually a higher score because the judge was awake, was paying attention, and was more knowledgeable while the 9.3 judge had not had their coffee, was daydreaming, and actually had less experience. What makes the judges ratings robust is having enough judges and aggregating their scores. Essentially this is the way the price mechanism is supposed to work if it has 'good enough' information (enough subjective perceptions of value that are aggregated and responsive to continual changes).
The idea of ubiquitous Big Data is that whole new aggregations of rated values/measures become available.
The next couple of readings help flesh this out. First is an interesting and sort of antidote to our enamourment with numbers and point out the vulnerability to the many potential consequences of our reliance on measures - where the cult of "If it can't be measure it does not exist," meet "every measure which becomes a target becomes a bad measure." Although this is and the following are a bit dense they are well worth the time to grasp their meaning.
Gilles PaquetA few quotes:
Optimum Online. Vol. 39, Issue 1, Mar 2009
Optimum Online. Vol. 39, Issue 1, Mar 2009
Quantitative methods have been used from time immemorial as a powerful instrument of reasoning. The problem arises when the use of such tools becomes the basis of a cult roughly captured by the motto “if it cannot be measured, it does not exist.” Such a cult distorts the appreciation we have of socio-economic phenomena, and this mental prison acts as blinders that have toxic, unintended consequences for public policies when they are shaped by an apparatus thus constrained…..
Another important cost of the quantophrenic cosmology is the so-called Goodhart effect – a phenomenon akin to the Heisenberg principle that suggests that quantifying transforms the world it tries to measure. Hoskin (1996: 265) has suggested a formulation of the Goodhart effect along the following lines: every measure which becomes a target becomes a bad measure. This is so because the calculative fantasies of managerialism transform the environment into which they are introduced. Individuals and organizations come to think of themselves as auditees, and quantification distorts the character of the universe to which it is applied (Shore 2008): its effects are irreversible and generate a fixation on the metric rather than on the creativity and initiative that any practice requires. Getting results may be the explicitly stated goal mentioned, but numerology transforms the very notion of what the goal is, of what the organization is about…….
Following up on the reference Gilles Paquet make to Hoskin - the following is an excellent article that reveals a great deal about our relationship with number, accounting and strategy. Hoskin and Frandsen's article
Elaborates the origin of writing in the first human insights in how to keep measures of the products of agriculture. What is interesting is, although human's were proto-farmers for about 30,000 years they only became real farmers within the the last 10,000 years (or so). The key may have been the development of the capacity to 'measure' stuff, which finally enabled humans to develop the social-economic structures for exchange. The following quotes introduces the paper (this paper is well worth the read).
Given the way that strategy is no longer conceived in many organizations and settings as purely a preserve of top management or reserved to the category of ‘strategic planning’, the question ‘where is strategy?’ has a new significance. Clearly strategic planning remains integral to strategizing, but as entities claim more and more to be knowledge-based or learning organizations, strategizing is increasingly understood as needing to be disseminated down and out through entities, and within increasing numbers of their members (i.e. human subjects). This constructs new roles for accounting in relation to strategy as increasing numbers of organizational members become constructed not purely as ‘calculable subjects’ (cf Foucault, 1975), but also, as managers and strategists have always been, as ‘calculable and calculating subjects’, and in this capacity even as
In this paper we wish to consider the relations of accounting to strategy in two ways which we contend are interconnected. First we want to suggest that this new kind of linkage of accounting to strategy is an extension of a principle which has been visible, if not always noticed, from the emergence of modern strategy in the context of the invention of modern management and the entity that Alfred Chandler (1977) attached to that invention, the ‘modern business enterprise’. Strategy in its modern and distinctive forms has always been a product or precipitate of the new kinds of (cost and financial) accounting developed as a key means of enabling management to be developed in the first place. Strategy as practiced since then has been what Chris Smith has called in the modern context ‘Strategy as Numbers’ (Smith, 2003).
But further, what we are witnessing, as exemplified in two contemporary settings we consider here, is the extension of this principle both across the extending expanse of – and down within every increasingly subdivided fraction of – organizational ‘space’ and ‘time’, something increasingly achieved by the construction (including the selfconstruction) of all of us as populations of calculating/calculable selves whose forms of calculation take shape under the sign of accounting. Thus organizationally and individually we have to answer the question, “where is strategy?”, not only by seeing it as currently or synchronically disseminating out and down in organizational space/time, but by understanding that it increasingly does so, for reasons that can be understood diachronically, as numbers, and particularly as those accounting numbers which always name and count, and so ‘value’.
The next articles in this line of thought illuminates the issues of design. The digital environment represents a wicked design problem - not simply a technological design problem but a challenge of designing measure - which are the 'environment' of evolving new cultural structures and institutions.
Jonathan Blow, is an icon in the Game Development world. Adored my many independent game developers and sort of vilified by many commercial game makers. This is a rather long video - but is one my son recommends as the most accessible to people who aren't into video games, and one of the best.
How are games related to this stream of thinking about Big Data and the digital environment? I think that the framework that I think is best for understanding the trajectory of video games is that they are functions as the world's most profound laboratories for experimentation with 'Governance Policy' that can shape and harness people's motivation and passion. They are the laboratory for exploring the conditions of the future of working in the digital environment. I like to think of game developers as designers of social-political-economic systems - who must also listen and motivate their citizens.
Blow is concerned with developing new types of games, ones that are less about generating and exploiting 'addictions' and more about a deeper, healthier engaged experience of self-expression/discovery. This seems to me to be key, to the creation of an innovative organizational culture. A culture not 'gamified' for productivity of a machine architected organization, but of an enterprise that enables the self-organizing of complex systems energized by positive human intrinsic passion and will.
This is the quote describing Jonathan Blow's presentation:
Video games have evolved tremendously over the past few decades; they're much more entertaining than they used to be. That is not by accident; we, the community of game designers, have been continuously refining our techniques. The most common way we do this is by testing out our games on you, the players, and optimizing for the "best" result (where "best" is defined by us). As this process is ongoing, what kind of relationship exists between the designer and the player? Is it artist/audience, experimenter/subject, entrepreneur/customer, or tycoon/resource? Invariably it's some admixture of these things, the particular ratios for a given game being chosen by its designers (usually without awareness that a decision is being made). Today, due to the way the Internet is widely used, and because game designers are becoming more serious about certain aspects of their craft, the iteration time of this game design optimization process is shorter than ever before: designers can observe their players much more thoroughly, and more quickly, than they ever have in the past. At some point a quantitative change becomes a qualitative one: the result of all this competency may be heavily destructive. Some aspects of the current notion of "good game design" may in fact be very bad, or at least indefensible, from an ethical standpoint. Today's "better" video games spend a great deal of effort to undermine defenses that took you tens of millennia to evolve. They tend to be successful at this. As designers keep evolving their craft and gain greater analytical power, what will happen?
The next link follows from all the others and is a natural complement to Jonathan Blow's video. This is a great blog about Janet Murray's book on Design. Which I think is the master discipline to enable a functioning digital environment, one that is human centric. Murray outlines four fundamental affordances that are the key to the digital medium - procedural, participatory, encyclopedic, and spatial.
I think these are well worth considering. However, I would re-conceptualize what she calls procedural as 'Programmable' and would call the spatial 'Time-Spatial'. Despite this nitpicking on my part - I think what she offers is a foundation for the design of institutions, organization, services, things, etc. in the digital environment. Again this is well worth the read.
The following is a quote from the article:
By Gerd Waloszek, SAP AG, SAP User Experience – September 6, 2012
In her 2011 book Inventing the Medium – Principles of Interaction Design as a Cultural Practice, Janet H. Murray outlines a new (she calls it "innovative") framework for design, in which she adopts a humanistic instead of a social science or industrial design perspective. Murray regards this framework as a unifying framework for many disparate but related efforts and suggests that it helps speed the process of innovation because of its focus on core human needs and its intention to "focus designers' work on useful and lasting innovation" instead of on mere novelty.
Murray explains that digital designers have two responsibilities: "To create the artifact that best serves the needs of people who will interact with it, and to advance the digital medium as a whole." In her view, designers are engaged in a "collective design process," performing "the collective cultural task of inventing the digital medium," in other words, expanding "the meaning-making conventions that make up human culture" and "our ability to understand the world and to connect with one another." Read my review (in progress) of Murray's book for a closer look at her humanistic design perspective.
In this article, which complements that review (in progress), I want to briefly outline this approach and its humanistic design perspective because it differs considerably from typical HCI-oriented design approaches that currently dominate the UI design field. Please note that there is a certain overlap with the review.
One last link that points to one dot in the matrix of the unfolding digital environment. Google and other's are working around ever more precise measures of time - the global synchronization of data that can turn the world into a single unified database.
Here's the initial paragraph:
At a symposium in the mountains of Montana, Jeff Dean — one of Google’s most important engineers — revealed that the web giant was working on something called Spanner, describing it as a “storage and computation system that spans all our data centers.” He said the plan was to eventually juggle data across as many as 10 million servers sitting in “hundreds to thousands” of data centers across the globe.
The scope of the project was mind-boggling. But Dean provided few details, and it wasn’t clear whether Google was actually using the platform in its live data centers. Then, on Tuesday, the paper hit the web.
This week, as reported by GigaOm and ZDnet, Google published a research paper detailing the ins and outs of Spanner. According to Google, it’s the first database that can quickly store and retrieve information across a worldwide network of data centers while keeping that information “consistent” — meaning all users see the same collection of information at all times — and it’s been driving the company’s ad system and various other web services for years….
The wicked problem is harnessing the best human have to give. It involves stewarding conditions and cultures that align people's intrinsic (and extrinsic) motivations with their experiences. With ubiquitous Big Data - made visible and available with mobile technology (including Google Glasses - which will do to visualizations of data what Google Search did to our assumptions of asking questions whenever and where ever we want), we return to the McLuhan quote at the beginning. Our whole histories of interaction will collapse into an eternal present - as the foundation of a new social currency - that enables new forms of exchange necessary in the world of hyper-connection, networked individualim, responsible autonomy, and collective intelligence.
Our ability to see our own trajectories (and those of everyone else) through the digital environment requires new concepts of 'privacy' (as the right to not be interfered with), and transparency that collapse the past into the present. The medium of exchange become our capability and our reputations.