This stuff isn’t really news, so I’ll put it below the fold. It’s for those of you who might be interested in seeing how secondary analysis of survey data has changed over the past fifteen years.
Check out this graph of “house effects” (that is, systematic differences in estimates comparing different survey organizations) from the 1995 article, “Pre-election survey methodology,” by D. Stephen Voss, Gary King, and myself:
(Please note that the numbers for the outlying Harris polls in Figure 1b are off; we didn’t realize our mistake until after the article was published)
From the perspective of fifteen years, I notice two striking features:
1. The ugliness of a photocopied reconstruction of a black-and-white graph:
2. The time lag. This is a graph of polls from 1988, and it’s appearing in an article published in 1995. A far cry from the instantaneous reporting in the fivethirtyeight-o-sphere. And, believe me, we spent a huge amount of time cleaning the data in those polls (which we used for our 1993 paper on why are campaigns so variable etc).
3. This article from 1995 represented a lot of effort, a collaboration between a journalist, a statistician, and a political scientist, and was published in a peer-reviewed journal. Nowadays, something similar can be done by a college student and posted on the web. Progress, for sure.