Fake polls. Amateur-ish polls. They’re becoming a problem, and they’re likely to become a bigger one. But you don’t need to be a statistician to spot a suspicious poll. I’d recommend a few simple questions that everyone, professionals and amateurs, can ask of any poll to help avoid the suspect ones. And almost everyone, including me, has been fooled. (Almost the entire political media, for instance, was reporting on polls from Research 2000 before it was unveiled as a forgery.) There are going to be legitimate pollsters that don’t meet all of these criteria. But if you come across a “pollster” that fails most of these tests, I’d take your mouse off the retweet button and go on with your day.
- First and foremost, does it seem professional? That may seem too basic, but it works surprisingly well. Is a pollster’s press release riddled with typos? Reputable pollsters are run by publicly identified people, and if they’re putting their professional reputations on the line, they probably want to make a good first impression. Spelling simple words wrong or misspelling the candidates’ names is often a sign that either a pollster doesn’t know what it’s doing or isn’t on the level. Small mistakes usually come with big mistakes.
- Who? Who conducted the poll? Does the pollster have a long track record? Check out the polling firm’s website — are there real people with expertise listed there? Does the pollster even have a website and not just a Twitter account? (Websites are pretty easy to create, but some fake pollsters don’t even do that.) If a pollster doesn’t reveal the people working for the company, then you probably don’t want to cite the firm’s numbers.
- How? How was the survey conducted (e.g., via automated phone, live telephone interview or on the internet)? If it was on the internet, see how the pollster was getting people to participate in its polls (e.g., via its own panel or Google Surveys). If it was on the phone, find out which phone bank was doing the calling. If a pollster isn’t revealing its methodology, don’t trust it. Legitimate, professional pollsters prize transparency.
- What? What questions are being asked? If it’s a poll about an election, legitimate pollsters will typically ask respondents more than simply who they prefer, Candidate A versus Candidate B. The pollsters will want to find out why people are voting the way that they are (what issues matter to them, for example, or how favorably respondents view the candidates). At a minimum, pollsters will ask demographic questions in order to weight their data properly. If a pollster isn’t revealing this data and how it’s being weighted, be suspicious.
- When? This works two ways. First, when was the poll itself conducted? And how many people did it reach? Those are crucial, standard details every on-the-level pollster releases. Second, when was the polling company founded? If there’s no answer, be suspicious. If it was only very recently, treat its results with caution until it has a body of work to judge.
- Why? Polls cost money, so most pollsters aren’t conducting them on a whim. Academic institutions often poll to increase their name recognition, or to provide students an educational opportunity. Most professional pollsters conduct surveys to make money. If there isn’t something on the website that tells you why the pollster is conducting the poll, something is probably up.
- Where? Find out where the company is located. Even in the age of the internet, most pollsters have a physical location. An address that you should be able to send a piece of mail to. An actual place that you can check exists via a website like the Whitepages.
Can you reach the pollster? Some fly-by-night operations won’t even have phone numbers on their websites for you to call. That’s probably not a good sign. If there is a phone number, see if it’s toll-free (
costs more money to the company, but less to the consumer). If it’s not a toll-free number, see if the area code matches the area where the company is located. And if you’re really adventurous, pick up a phone and see if you can speak to a real person. (You can also try the “Shattered Glass” trick, if you’re suspicious.) If there’s no number, shoot the pollster an email (assuming its website includes an address). Do you get a response?
- Short on time? Check to see if polling websites like HuffPost Pollster or FiveThirtyEight have cited the pollster. If they haven’t, there’s probably a good reason.
- Still unsure? If you think there’s a fake poll out there, simply email FiveThirtyEight at firstname.lastname@example.org. We’ll look into it.
Read more: Fake Polls Are A Real Problem