 |
| Magritte |
Who do you trust? I have asked this question many
times and I ask it of myself each day. In this continuation of best books of
the past year I will attempt to tie together a few of the connections I have
attempted to make between those who I have learned to trust about not trusting
ourselves and others.
I have an apology to make. For quite a while now I
have been repeating a mantra on this blog, on many forums and websites, and in
person to all who happened to be in range of my voice: “without data, you are
just another person with an opinion”(originally attributed to Andreas
Schleicher).
After reading Nate Silver’s The Signal and The Noise; why so many predictions
fail--but some don't, I have learned that I have been giving out bad
advice. So let me rephrase my mantra, without data everything is just an
opinion; with data everything may or may not be based on the incomplete
assumptions of the researchers or the data interpreters.
If my update sounds like a legalism, it isn’t.
Silver has convinced me, through data and stories, that much of what we assume
to be true because of statistics and data, isn’t. Or isn’t useful. Or if
useful, it is useful because designed to create an outcome that is desired
rather than an objective assessment of a problem or situation.
In his conclusion to what I consider to be an
invaluable book, Silver gives a list of statements:
1.No investor can beat the stock market.
2.
No investor can beat the stock market over the long term
3. No investor can beat the stock market over the
long run relative to his level of risk.
4. No investor can beat the stock market over the
long run relative to his level of risk and accounting or his transaction cot
5. No investor can beat the stock market over the
long run relative to his level of risk and accounting or his
transaction costs, unless he has inside information.
6. Few investors beat the stock market over the
long run relative to their level of risk and accounting for their transaction
costs, unless they have inside information.
7. It is hard to tell how many investors beat the
stock market over the long run because the data is very noisy but we know that
most cannot relative to their level of risk, since trading produces no net
excess return but entails transaction costs, so unless you have inside
information you are probably better off investing in an index fund.
He then
summarizes: “The first approximation—the unqualified statement that no investor
can beat the stock market—seems to be extremely powerful. But by the time we
get to the last one, which is full of expressions of uncertainty, we have
nothing that would fit on a bumper sticker. But it is also a more complete
description of the objective world.” Silver essentially asks us to
distinguish between what a variety of writers have called some form of the
distinction between thinking fast and slow (Daniel Kahnemans’ term as well as the
title of his wonderful book). Humans are wired to want quick and easy
descriptions. It is evolutionarily advantageous to do so in the world of the
savannah of tens of thousands years ago. But while the wiring is still in
place, the world we live in is far more complex. And a great deal of the
complexity comes about due to the overwhelming amount of data that we are
confronted with each day. Today, juts today, there will be more data generated
than virtually all the data ever collected in the history of humankind until
just a few years ago. (Sliver and many others on TED and elsewhere underscore
this data on data). But we still look for the simple solution. Or if we do not
look for the simplest solution, then we look for a small amount of data that
will lead us to a simple solution. It is this cause and effect action on our
parts which makes so many predictions seem laughably bad just a few minutes,
months or years down the road. And tracing how this might be avoided is the
theme of this book.
But let me start with a paradox. Silver is a
McArthur genius grant winner for his successful work with data. His analysis of
baseball stats and presidential and congressional elections is, for those who
follow this kind of stuff, legendary. And yet, the man known to be a data
genius has written a cautionary tale. Anyone reading it will come away thinking
that what he or she knew they knew as true may not be. In fact, it may be
harmful to anyone who bets resources on a business, or an idea, or an ideology.
I use the word ‘bet’ for a purpose. Silver has some
great words on how professional gambles make it big or fail. And he should
know. Silver spent some time as a professional gambler and reveals his secrets
and his reasons for giving up this pursuit. He learns a cautionary tale and he
seems to come out in favor of what another Peter Sims calls 'little bets.'
Little bets assume we do not have enough knowledge or accurate data to go all
in with any frequency. Instead, we should spread ideas and resources out to discover as many alternatives and solutions we can. The odds are almost always
against a single big win; the odds are much better when multiple hands are played. But as Silver would point out, not always. And this uncertainty is
what he demonstrates and then what he advises us to carry around with us inside
our heads.
We need to be suspicious of unsupported
information, but we need to be almost equally suspicious of decisions made with
incomplete data, and virtually all data is incomplete.
Much of what Silver has to say is in part based on
the tutelary god of statistics: Thomas Bayes. Silver devotes much time
demonstrating how Bayes's theories on data should be the starting point
for all who hope to use data to make important decisions of any sort. I am not
adequately educated to go into an analysis of the statistical probabilities of
Bayes’s work, but I can come to the conclusion, based on my own experiences
with data, that what Bayes’ work demonstrates is that the statistics we use are
largely guided by incompleteness and bias:
As an empirical matter, we all have beliefs and
biases, forged from some combination of our experiences, our values, our
knowledge, and perhaps our political or professional agenda. One of the nice
characteristic of the Bayesian perspective is that, in explicitly acknowledging
that we have prior beliefs that affect how we interpret new evidence, it
provides for a very good description of how we rec to changes in our world. P
258
As an avowed Rortian pragmatist, I think the word
‘description’ is apt and useful.
Statistics are just numbers after all. It is the stories, the descriptions, we
have before the numbers are in front of us, that affect our reading of the
numbers and then affect significantly the stories we then tell to others once
we have ingested the numbers in a way that can never be objective or complete.
We are, as Jonathan Gottschall says, story telling animals. And as Rorty so
often said, it is our ability to persuade people through words that makes us
what we are and determines what we do. Truth has nothing to do with it. So
Silver, citing Bayes, demonstrates that the experts with data are all too often
flat out wrong. Whether it is pundit on TV, or ‘experts' on the Cold War (also
cited by in a good book on this topic), or in any other field, the experts are
experts because we call them experts and they believe themselves to be experts.
All too often, the data behind the title proves otherwise.
And now it is time to say why I think this point is
essential, at least for me. I have been giving out bad information based on
incomplete data. Here is just one issue in which I was swayed by rhetoric
rather than data. Others, in which I was flat out wrong, will be the subjects
of forthcoming entries.
The first is the most personal in several senses of
the word. For many yeas I have been giving lectures and appearing in various
media to share my supposed ‘expertise’ on the topic of college admission
essays. Many of the entries on this site are devoted to questions about essays,
most of them focused on essays written for highly selective colleges and
universities.
While I still think my training as a writer and my
experience in the field permits me to say some things about writing great
essays I am now not sure I have the data to continue to tell students, at least
in any general way, whist will make a good essay for the purposes of gaining
admission to a selective school. Why? Starting this summer I have been doing
some research on responses to essays. I have posted essay on my blog, on
College Confidential and on quora.com. I asked for feedback and got some
surprising results. In a number of cases, the responses from ‘experts’ were
almost diametrically opposed in whether the words on the page were great or
terrible. Some of the experts hated the essays. And yet some of these essays
were significant actors in the students be offered admission to Princeton,
Yale, Harvard, and Stanford. This taught me a coupe of things. First, I thought
that great essays could be interpreted as such by anyone who is supposedly an
‘expert’. Silver taught me that the experts, like political pundits or academics
whose expertise are foreign affairs, frequently get things totally wrong.
In addition, the responses to the essays led me to
do some research on who are at the front line of gatekeeping at many colleges
and universities. In the old days, the teaching faculty had a significant role
to play in evaluating applicants to schools. No longer. Now the vast majority
of readers of applications are recent graduates with degrees in higher ed. or
in subjects often having little to do with the written word. On top of this
most schools do not give an intensive training course in how to read essays.
They do train people how to read applications but the essay portion is not
given any intensive scrutiny in terms of testing readers on what they think a
great essay might be. On top of this, the kinds of essays I like and promote
are risky. In advice I gave for the US News college issue on writing essays I
told students to take risks in no uncertain terms. “Take A Risk”. Period. I
have been trained and have read enough from CEOs and other people at the top,
that innovative risk-takers are the ones they want and the ones who will
succeed at the highest level. But for me this often means stylistic risks.
Creative non-fiction is the fastest growing field in creative writing
departments these days and yet, if my research means anything, most of the
people reading admission essays have little or no familiarity with the form.
George Saunders or David Foster Wallace, or even Constance Hale write wonderful
prose, but in ways that are not exactly in line with typical educational
essays. So some risk takers will be hurt by being creative.
And finally, now that the need is for STEM people is
being foregrounded, an essay that covers an interest or passion in medical
research or engineering may not need a Faulknerian touch. To sum this up, there
needs to be a great deal more context about essays before any general advice is
thrown out to the masses. Like Silver’s movement from a flat out assertion
in step 1 to a much more muted step 7, there is no general rubric for writing
an admission essay that is actually all that useful. Each essay needs to be
contextualized and if not, then following a how to in some book or essay on
essays may have negative consequences. I will be addressing this issue in much
more detail over the coming months, but I certainly learned that my easy short
answer of ‘take a risk’ sounded good in a speech or the beginning of an essay,
but it did not actually have enough data behind it to prove it useful for a
number of people.
Lady Gaga meets Julius Caesar
Let me end my review by returning to Silver: “If there is one thing that
defines Americans—one thing that makes us exceptional it is our belief in
Cassius’s idea that we are in control of our own fates.” The Signal and the
Noise is Shakespearean in its reach. I don’t mean the prose is so lofty that
plays will be made of it. Instead, Silver is aware, like Shakespeare, that
hubris usually leads to a steep fall. Julius Caesar ignores data in
Shakespeare’s play and things do not go well for him. Both Shakespeare and
Silver underscore there is far too much data and we are far too self-deceiving
for us to lead lives in control of our fates. I think he has proven his case.
The last words of his book are ones I hope to keep in my head each day: to be
“a little more modest about our forecasting abilities, and a little less likely
to repeat our mistakes”. It is no accident that Silver ends with the word ‘mistakes’,
those things we can learn from if we take the time to learn why we erred.
Silver himself is modest about his many talents, but the book is one of the
best of many I have read (for example, Dan Garner’s FutureBabble: Why Pundits are hedgehogs and foxes know best), that ask us to look inward at our preconceived
ideas and outward at the complexities that comprise the world.
No comments:
Post a Comment