#BigData: From Private to Public Consumption Part 1

It’s impossible to surf the Internet these days and not come across articles about “big data,” the massive amount of information we post online and that government agencies and corporations collect. Clicks, purchase history, the latest “like” on Facebook or +1 on Google+ are all examples of the types of big data being collected. Whatever the interaction, it gets recorded and stored for future reference. And with calls for “Do Not Track” legislation back on the table, there is a level of awareness not previously present.

What is forgotten is that we have been interacting with big data all of this time, even before the advent of the “like” and +1. 

We’ve been interacting with big data any time we have set foot in a Walmart.

In his book, The Wal-Mart Effect: How the World’s Most Powerful Company Really Works — and How It’s Transforming the American Economy, Charles Fishman points out that, because of Walmart’s decision to eliminate the cardboard packaging of deodorant,

Whole forests have not fallen in part because of the decision made in the Wal-Mart home office at the intersection of Walton Boulevard and SW 8th Street in Bentonville, Arkansas, to eliminate the box. The nickel savings may seem trivial, until you do the math. With two hundred million adults in the United States, if you only account for the nickel on the container of deodorant in the medicine cabinet right now, that’s a savings of $10 million, of which customers got to keep half, $5 million, just for one small change, unnoticed by consumers, more than a decade ago.

Read that again.

Whole forests saved. Nickels adding up to $10 million in savings.

All saved by simply eliminating one aspect of deodorant packaging, its cardboard container. And Walmart used its vast treasure trove of data, its “big data,” or what Fishman refers to as its “insight, and its muscle,” to institute a sweeping industry-wide change.

And its treasure trove of big data is not limited to its home office. Walmart makes it available to its suppliers via its proprietary system, called Retail Link.

As Fishman explains it, Retail Link “contains a record of every sale of every individual item at every Wal-Mart store, every hour of every day for the last two years.” Its suppliers, therefore, can easily track sales and “understand where, and when, and why their products sell at Wal-Mart.” In fact, “The data that pours out of Retail Link for even modest-sized suppliers is so enormous, so complicated, and so rich that companies now specialize in teaching vendors how to mine the data for insight.” Tracking technology has advanced so significantly since Retail Link was made available to Walmart suppliers in 1991, that it now requires special training to mine all of that data.

And all of that is data we have volunteered simply by standing in line at a Walmart checkout counter, waiting for items to be scanned. And Walmart “keeps track of the number of items per hour each of its checkout clerks scans at every cash register at every store, in every state, for every shift as a means of measuring their productivity.”

That’s not much different from tracking your time spent drafting a motion or responding to client communications. The difference is the access to big data.

Access to big data will be discussed in Part 2 of #BigData: From Private to Public Consumption. We’ll look at examples of how the public used to access big data, and how the development of tools and technology has enabled the shift from private to public consumption. Stay tuned!