A review of Big Data: A Revolution That Will Transform How We Live, Work, and Think, by Viktor Mayer-Schoeneberger and Kenneth Cukier
@@@@ (4 out of 5)
While Edward Snowden bounces from one temporary refuge to another in search of safe harbor from the long arms of the U.S. government, the American public is starting to wake up to the reality of Big Data. The National Security Agency, long one of the pioneers in this burgeoning but little-appreciated field, has been teaching us — or, rather, Snowden, The Guardian, and the Washington Post have been teaching us — about the power that resides in gargantuan masses of data. Now here come Viktor Mayer-Schoeneberger and Kenneth Cukier with a new book that goes far beyond the headlines about espionage and invasion of privacy to give us an eminently readable, well-organized overview of Big Data’s origins, its characteristics, and its potential for both good and evil.
When we think of Big Data, we, or at least most of us, think of computers. However, the authors persuade us that the fundamentals of Big Data were laid down more than a century before the invention of the microprocessor. They point to a legendary American seaman named Matthew Maury. In the middle of the 19th Century, after 16 years of effort, Maury published a book based on 1.2 billion data points gleaned from old ships’ logs stored by the Navy that dramatically reduced the distances (and, hence, the time elapsed) in ocean voyages by both military and commercial ships. Maury used facts derived from decades of mariners’ observations to dispel the myths, legends, superstitions, and rumors that had long caused ocean-voyaging ships to pursue roundabout courses. Not so incidentally, Maury’s work also facilitated the laying of the first transatlantic telegraph cable.
If not the first, this was certainly an early application of Big Data, which the authors describe as follows: “big data refers to things one can do at a large scale that cannot be done at a smaller one, to extract new insights or create new forms of value, in ways that change markets, organizations, the relationship between citizens and governments, and more.” For example, if Maury had had available only a fraction of the old ships’ logs he found in the naval archives, his task would have been impractical, since each individual log doubtless included small errors (and an occasional big one). Only by amassing a huge store of data did those errors cancel out one another.
Now, in the Digital Age, the volumes of data that can be harnessed are, at times, literally astronomical. “Google processes more than 24 petabytes of data per day, a volume that is thousands of times the quantity of all printed material in the U.S. Library of Congress.” AT&T transfers about 30 petabytes of data through its networks each day. Twenty-four or 30 of something doesn’t sound like much, unless you understand that a megabyte is a million bytes, a gigabyte is a billion bytes, a terabyte is 1,000 times the size of a gigabyte, and a petabyte is 1,000 times the size of a terabyte. That’s 1,000,000,000,000,000 bytes. That’s a lot of data! But even that’s only a tiny slice of all the data now stored in the world, “estimated to be around 1,200 exabytes.” And an exabyte (I’m sure you’re dying to know) is the equivalent of 1,000 petabytes. So, 1,200 petabytes could also be stated as 1.2 zettabytes, with a zettabyte equal to 1,000 petabytes, and I’ll bet that not one person in a million has ever heard of a zettabyte before. Had you?
All of which should make clear that when we talk about Big Data today, we’re talking about really, really big numbers — so big, in fact, that almost no matter how messy or inaccurate the data might be, it’s usually possible to draw useful, on-target insights from analyzing it. That’s what’s different about Big Data — and that’s why the phenomenon is bound to change the way we think about the world.
We live in a society obsessed with causality. We often care more about why something happened than about what it was that happened. And in a world where Big Data looms larger and larger all the time, we’ll have to get used to not knowing — or even caring much — why things happen.
“At its core,” write Mayer-Schoeneberger and Cukier, “big data is about predictions. Though it is described as part of the branch of computer science called artificial intelligence, and more specifically, an area called machine learning, this characterization is misleading. Big data is not about trying to ‘teach’ a computer to ‘think’ like humans. Instead, it’s about applying math to huge quantities of data in order to infer probabilities: the likelihood that an email message is spam; that the typed letters ‘teh’ are supposed to be ‘the’; that the trajectory and velocity of a person jaywalking mean he’ll make it across the street in time [so that] the self-driving car need only slow slightly.”
The authors refer to data as “the oil of the information economy,” predicting that, as it flows into all the nooks and crannies of our society, it will bring about “three major shifts of mindset that are interlinked and hence reinforce one another.” First among these is our ever-growing ability to analyze inconceivably large amounts of data and not have to settle for sampling. Second, we’ll come to accept the inevitable messiness in huge stores of data and learn not to insist on precision in reporting. Third, and last, we’ll get used to accepting correlations rather than causality. “The ideal of identifying causal mechanisms is a self-congratulatory illusion; big data overturns this,” the authors assert.
If you want to understand this increasingly important aspect of contemporary life, I suggest you read Big Data.
Viktor Mayer-Schoeneberger and Kenneth Cukier come to the task of writing this book with unbeatable credentials. Mayer-Schoeneberger is Professor of Internet Governance at Oxford University, and Kenneth Cukier is Data Editor at The Economist.
Ten Big Issues Washington Is Ducking
This is the time of year when most of us record the New Year’s resolutions that will load us with guilt throughout the year because we never follow through with them. So, for a change this year, I decided to take stock not of my own life but of the state of our nation. What follows is my best effort to list (in no particular order) the ten most significant issues that the White House and the Congress should be addressing – but aren’t, and maybe never will. I write in the wake of a long-delayed compromise between the two parties, a deal that nobody likes and that, in its superficiality, illustrates just how far off the mark our elected leadership has strayed.
1. Public corruption
The dominance of money in politics is the root cause of much that ails us. Massive campaign spending, combined with lavish lobbying efforts, is largely responsible for corporate welfare, our shockingly inequitable tax code, the dangerous bloating of the financial sector, and the corporate dominance of the news media. It’s also a major factor in the country’s continuing dependence on fossil fuels. Every one of these issues cries out for systemic change, but in a society where the U.S. Supreme Court’s outrageous Citizens United decision holds sway, it’s difficult to see how any meaningful change can be enacted. The source of the problem lies deeper than policy, in the values that corporate money has sold to the public – at heart, the delusion that freedom means independence from government oversight, that society offers a level playing field to all comers, and that success can only be fairly rewarded if the winners take all. In The Self-Made Myth (reviewed here), Bryan Miller and Mike Lapham expose this value set for the illogical and self-serving approach that it is.
2. Military overreach
The United States spends more than $700 billion annually on what is characterized with Orwellian skill as “defense.” This amount is reportedly greater than the combined military expenditures of all the rest of the nations on Earth and is certainly larger than the total spent by all our potential adversaries combined. It’s also mostly money that could be so much more productively invested in advancing our true national security – upgrading our educational system, restoring our once-undisputed lead in science and technology, combating global poverty, and tending to our long-neglected public infrastructure. The late Chalmers Johnson’s Blowback series – Blowback, The Sorrows of Empire, and Nemesis – illuminate the extent of U.S. military overreach, and the steep price we pay for the dubious privilege of maintaining nearly 1,000 military bases around the world. We put Imperial Rome to shame.
3. Secrecy in government
Most of what we read about secrecy in our federal government concerns the “classified” documents such as those unearthed by Wikileaks not long ago or the information turned up by investigative reporters, often after years of pursuing Freedom of Information Act (FOIA) lawsuits. Sadly, hiding mountains of written records behind a cloak of secrecy, reprehensible though it is, should be the least of our concerns. Far more threatening to our liberties and our future as a democratic nation are the top-secret operations of the National Security Agency, the CIA, and the Special Forces, as well as numerous other activities carried out both at home and abroad in our name under the veil of black budgets for agencies that have never seen the light of day or through seemingly innocuous contracts with private companies. The Washington Post’s Dana Priest and William M. Arkin did a spectacular job of reporting about this tragically overlooked phenomenon in Top Secret America: The Rise of the New American Security State (reviewed here).
4. Overspending on healthcare
The U.S. currently spends an unsustainable 17% of GDP on healthcare – about one-half more than the second-highest spender in the world (Switzerland, at 11%). Americans frequently brag that we have the finest healthcare system in the world, but that’s true only for those who can afford to pay millions for the most advanced care when a health emergency strikes. Ours is the world’s most expensive healthcare system, not the best. Most of the rest of us would be far better off in France or some other industrialized country where government covers all costs and negotiates fair prices with pharmaceutical companies and other healthcare providers. And all the current talk about “reining in the deficit” is so much pointless chatter without two straightforward policy changes that could make a truly big difference: a drastic reduction in the Pentagon budget, of course, and adopting Medicare for All, otherwise hideously labeled “single-payer healthcare.”
5. Mass incarceration
One of my greatest disappointments with the Obama Administration is its continued prosecution of the so-called War on Drugs, the congeries of policies, police practices, and court decisions that has resulted in locking away more than two million Americans and subjecting our inner cities to a profoundly racist police regime. Michele Alexander’s landmark study, The New Jim Crow (reviewed here), lays bare the startling dimensions of these problems and their deeply rooted origins in the politics of the Nixon, Reagan, and Clinton administrations. That such policies could persist two generations after the Civil Rights Movement of the 1960s is abhorrent.
6. Global warming
Rarely do political issues rise to the level of existential crisis. Here’s one that does. As Mark Hertsgaard illustrates in Hot (reviewed here) through interviews with leading climate scientists, the scientific consensus about the impact of climate change has become more extreme with every new report – but has never caught up with the private projections of the most knowledgeable experts. Absent dramatic policy shifts on a global scale, which are unthinkable without strong U.S. leadership, it’s possible that Planet Earth will eventually become unlivable for the human race. We’re already destroying a million species a year, and climate change is compounding the problems caused by human encroachment on animal habitat. With or without human civilization, our global environment will be very different in the 22nd Century from what it is today – at a minimum, far less hospitable to homo sapiens.
7. The culture of violence
In the wake of yet another horrific mass murder that took the lives of so many innocents, public debate is focusing on such “solutions” as banning assault rifles and reducing the number of bullets permitted in an ammunition clip. Even if such measures could be written into law, which is unlikely, they would be laughably ineffectual. More than 9,000 people die every year of gunshots in the U.S. – rarely from assault rifles. Americans possess more than 200 million guns, most of them handguns, and can easily buy more at 51,000 licensed retail firearms dealers (compared to 36,000 grocery stores). None of this should be a surprise in a society that glorifies violence in film, television, video games, and comic books and obsesses about football, one of the most violent of contact sports. It’s time for America to grow up!
8. Chemical pollution
Most of the 9,000 or more synthetic chemicals now used in everyday products in the U.S. were introduced after World War II. Hundreds of them leave residues in our bodies with largely unknown consequences. (Only seven percent of “high-production” chemicals have been fully tested for toxicity.) In other words, we have been carrying on a dangerous biology experiment with our lives and our children’s lives for more than two generations. What we do know is that health problems that were once unknown or rare are becoming common, including asthma, reproductive abnormalities in infants, many forms of cancer, and autism. A simple idea – the precautionary principle – could address many of these unwanted consequences by stipulating that the burden of proof about the safety of any product falls on its producer. Nearly half a century after Rachel Carson’s Silent Spring, isn’t it astonishing that we should still have to make this argument?
9. A dysfunctional education system
For decades, it’s been widely recognized that many high schools are simply warehousing young people to keep them off the job market. Now it’s beginning to seem as though that’s the case with so-called higher education as well at many colleges and universities. When employers (myself included) complain that some recent college graduates can’t write or spell and either can’t read or simply choose not to do so, you’ve got to figure there’s some truth to these observations – and that there’s something fundamentally wrong with the way our country educates its youth. Whether the root cause is that schools teach the wrong things, that they teach in the wrong ways, or that the wrong people are doing the teaching is impossible to tell, but clearly the truth lies in some combination of these notions – dramatically compounded by our society’s failure to invest enough money to do the job right. Taking into account the number of hours that American teachers work, they’re paid far less than teachers in almost any other industrialized country. Shame on us!
10. A costly and dangerous food production system
An occasional outbreak of e coli infections or a newsmagazine exposé on the indiscriminate use of antibiotics in farm animals reminds us that all is not well with the way we Americans produce and procure our food. However, truth to tell, the scale and extent of the problem is far bigger than most of us understand. Ninety-nine percent of the meat we eat is produced in ways that are inhumane, ecologically unsound, and dangerous to our health. Our unrelenting hunger for meat is responsible for producing more greenhouse gases than all modes of transportation combined and is thus one of the single most significant factors in global warming. Pollution from factory farms is poisoning the water table in agricultural areas throughout the United States, and the dramatic overuse of antibiotics in farm animals that aren’t sick is exposing us all to ever more deadly antibiotic-resistant diseases. Jonathan Safran Foer’s recent book, Eating Animals (reviewed here), exposes these and other truths about our food production system.
If any of the above leads you to believe that I think the United States is in worse shape than other countries, you might consider the neglected issues I’d identify, say, in Bangladesh or Tanzania. If you don’t know from direct observation, take my word for it: they’re in far worse shape than we are.
Filed under Commentaries, FAQs & Commentaries
Tagged as citizens united, climate change, corporate dominance, corruption, current-events, defense spending, financial sector, FOIA, global warming, government, Jim Crow, money in politics, national security, Nonfiction, Pentagon, politics, public corruption