Dear John,
I don’t really have the time or the energy to get involved in another Big Fight right now, so I’m not going to respond in great detail to your long note to me at Huffington Post, other than to say that some parts would have benefited from a fact-check. But it was really only the first and most benign-sounding point that bothered me:
Don’t Create Standards You Will Find Hard to Maintain Yourself. You are hot right now – using an aggregate of other people’s work, you got 49 of 50 states right in 2008. I know how it is to feel exhilarated. I get the states right a lot too. But remember that you are one election away from being a mere mortal like the rest of us. We very good pollsters have missed some. They tell me you blew the Academy Awards and your projections in the 2010 U.K. elections were a tad squidgy. So be humble and continue to hone your craft. Be aware that some of your legions who adore you today and hang on your every word will turn their guns on you in a minute. Hey, I have been right within a few tenths of a percent – but you are a probabilities guy and even a 95% confidence level and a margin of sampling error are not enough for some.
Mr. Zogby, I think you may be mistaking me for my Wikipedia page. I don’t really spend a lot of time touting my accomplishments or resting on my laurels — there are no marketing materials of any kind on this site. I’m a process-oriented guy, not a results-oriented guy, because as you mention, there’s a tremendous amount of luck involved in making any sort of predictions. In the long run, if an unskilled forecaster gets something right 50 percent of the time, a skilled forecaster might get something right 55 or 60 percent of the time. There are very, very few exceptions to that, in politics or in any other discipline. So when we get something right, we usually just move on with our lives rather than brag about it. And when we get something wrong, we’ll usually do a post-mortem and try to figure out if we were unlucky or stupid, but not wallow in self-pity.
Now, I’m certainly not going to pretend that we take an attitude of austere academic humility toward everything that we do. We’re happy to engage both our friends and our critics in lively arguments, and we can be sarcastic and combative at times. I have a background in competitive, adrenaline-intensive disciplines like poker, policy debate, and sportswriting, and that attitude has become hardwired by now.
The only thing that I knowingly am a bit conceited about is the only thing that I have complete control over: the amount of effort that I put into FiveThirtyEight and my other projects. I work my butt off — 80-100 hour weeks have been the norm for about two years here. When we think something’s wrong, we’ll fix it; when we think the approaches that others are taking to a problem are inadequate, we’ll tackle it ourselves. I’m not an 80/20 guy: most people, when presented with a choice of two approaches to a problem, will take the easy road, the one that can get them 80 percent of the way to where they want to be for 20 percent of the effort. I don’t share in that philosophy at all. In our flat and interconnected world, differentiation is key, and we want to do something right (or failing that, differently and interestingly) if we’re going to do it at all.
Along those lines, I think you need to examine the thought process behind your interactive (Internet) polling, which any objective attempt at analysis will demonstrate has achieved vastly inferior results, beyond any shadow of a doubt. They don’t do justice to the years of solid work embodied in your live-operator polls.
Beyond that, I can’t speak to how you run your business, because I won’t presume to know very much about it: how you arrive at the decisions that you do, how strong your desire is to use the right means rather than merely hoping to achieve the right ends. With some pollsters dropping like flies, and others flooding the zone with cheap, high-volume polling, it is imperative that we have a diversity of opinions, and I hope that Zogby International will continue to be a major part of that.