Politicians and the public have an historically uneasy relationship, governed by two competing paradigms: (1) politicians ask the public what it wants, or (2) politicians tell the public what it wants.

For those of us who grew up in democratic societies, it’s easy to forget that the weight of historical precedent falls squarely on the second paradigm.  Popular sovereignty isn’t a new invention – the Greeks and Romans toyed with it millennia ago – but history shows us far fewer democracies than dictatorships.

There's more...

It shouldn’t surprise us, then, that polling is a relatively recent invention.  Back in the days of Plato and Aristotle, the Athenian city-state only contained about 30,000 citizens with the right to vote.  If I was an ancient Greek and I wanted to know how my fellow citizens felt about Socrates’ corrupting the youth of Athens, all I had to do was raise the question in a public forum.  No one would worry about demographics: the old rich white male vote vs. the middle-aged rich white male vote; the “Hill of the Muses” residents vs. the “Hill of the Nymphs” residents.  Demographic differences would be small and largely unimportant.

But in today’s America, and all across the world, taking the pulse of the people is a much more involved endeavor.  Does an auto mechanic in South Compton share the same views as an accountant from Bel Air?  Will a farmer from Kansas tend to agree with a professor at MIT?  And if not, how can we understand the patterns underlying how different people think about the world?

Polling, as we’re familiar with it, begins in Pennsylvania.  In July 1824, a newspaper called the Harrisburg Pennsylvanian surveyed about 500 potential voters (in Delaware, oddly enough) and found that Andrew Jackson held a commanding lead over John Quincy Adams (70% vs. 23%).  Jackson went on to win the popular vote by 38,000 – though, this being America, he still lost the presidency to John Quincy Adams.

The poll of 1824 was what we call a “straw poll”.  It involved no complex analysis of statistical trends.  It was just a vote.  (You may be asking yourself, “How is that different from any other poll – are they not like voting, then?”  We’ll get to that in due course, gentle reader)

Although the poll of 1824 failed to predict the vicissitudes of constitutional law, the utility of polling quickly became apparent.  Who among us wouldn’t like to know the results of an election before it happens?  And for over 100 years, straw polling was the name of the game.  Some polls performed well; others did poorly.  But over the next decades, the art of straw polling was refined until the Literary Digest premiered in 1890.

Literary Digest became the premiere polling group in America, correctly predicting the winner in each presidential election from 1916 to 1932.  Literary Digest had a very simple philosophy for polling: the bigger your sample, the more accurate your results.

1936 brought the contest between Franklin Roosevelt and Kansas governor Alf Landon.  The Literary Digest mailed out more than ten million questionnaires.  Yes. 10,000,000. They received over two million responses, and the week before Election Day, Literary Digest predicted an overwhelming win for Alf Landon.  The results of the Literary Digest poll were:

Landon, 1,293,669 (57%)
Roosevelt, 972,897 (43%)

At the same time, George Gallup was conducting a new type of poll – one that used a smaller sample, but tried for wider demographic representation.  Gallup’s polls showed a very different picture from the Literary Digest straw poll – so different, in fact, that Gallup predicted a victory for Roosevelt fully a year before the election.  In July of 1936, Gallup went a step further and predicted the outcome of the Literary Digest poll itself.  In both cases, George Gallup was right.  Roosevelt won a crushing victory, earning 63% of the popular vote and 46 of 48 states.  Only Maine and Vermont went to Landon.

How could Literary Digest be so wrong, with two million respondents?  The answer is simple, and entirely a product of how Literary Digest conducted their straw polls.

Literary Digest mailed out ten million questionnaires.  To accomplish this, they had to find address records for ten million individuals.  They did this by mailing questionnaires to all their own subscribers, then to individuals in the auto registration records and the phone books.  By self-selecting for individuals affluent enough to subscribe to magazines or own cars or telephones – during the Great Depression no less – Literary Digest was ignoring large segments of the US population.  For the population segments that looked like the Literary Digest sample, the 1936 poll was perfectly adequate.  For the electorate as a whole, the poll was woefully underrepresentative.

Since the development of statistical polling methodologies, there have still been errors.  Gallup blew it big-time in 1948, with their prediction that Dewey would handily defeat Truman.  But the nature of polling remains largely the same as it was in 1936.  Refined and improved, of course, but relying on the same statistical methodologies that let George Gallup correctly predict the outcome of the 1936 election.

This series, The Plebian’s Guide to Polls, will explore those statistical methodologies from a layman’s perspective.  We hope that, by reading, you can come to a fuller understanding of what polls do, how they work, and why they aren’t always accurate.  Please stop by again next Wednesday, when I’ll roll out the next segment in this series: “Why We Poll”.