UtahPolicy.com conducts regular political polling, most of which is greatly appreciated by the political community we serve. It’s important to have independent survey research on the key issues of the day and to measure job performance of elected officials.
Commenters on social media frequently criticize the research, questioning its accuracy and objectivity. It’s nice that we have elections so the accuracy of our research can actually be measured. The recent primary election showed that our pollster, Dan Jones & Associates, did a phenomenal job measuring the support of candidates in the major races.
The Jones’ research in advance of the primary had Mitt Romney defeating Mike Kennedy by 43 points. The survey showed John Curtis defeating Chris Herrod by 47 points. Guess what? Those are precisely the margins of victory in the actual election.
Such accuracy is remarkable given the difficulty of conducting survey research in today’s communications environment where many people do not have landlines and won’t answer cell phones unless they know the caller.
A survey in advance of an election will never have the exact percentages of election results. That’s because pollsters allow respondents to answer “don’t know” or “no opinion.” In actual elections, such an option doesn’t exist. So, in a survey what becomes the most accurate measure, when compared with actual results, is the point spread.
Usually, getting the point spread within the margin of error is considered to be extremely accurate. To get the exact point spread as the election result is amazing. Dan Jones and his team at Cicero Group deserve congratulations for the good work.
Many critics of polling don’t understand polling methodology. Some social media commenters say, “Well, they didn’t ask me.” Or, “how can you measure the opinions of millions of people with a sample of 600?”
As I’ve written previously, polling is only as accurate as the methodology used and the care and integrity of the pollster. Certainly, conducting accurate survey research is more difficult today than in past decades. But it can still be done with a high degree of accuracy.
Dan Jones & Associates is one of the oldest and most respected polling organizations in the country. Survey research firms don’t last very long if they aren’t careful and accurate. Clients won’t stick with firms whose research conclusions differ dramatically from actual results. Jones has been successfully conducting polls for many decades in Utah and other states.
Some research firms use pre-recorded automated calls that ask respondents to hit numbers on their phone to respond to questions. That polling methodology isn’t considered to be as accurate as live interviews. Respondents aren’t as likely to be truthful and it’s harder to get the demographics right.
Good pollsters today use a combination of land lines, cell phones and on-line panels to obtain an accurate random sample with proper demographic characteristics.
Some people question the accuracy of polling. Here’s an analogy that shows how good polling works:
Let’s say you have a barrel with 10,000 marbles in it. 2,000 are red; 5,000 are blue; and 3,000 are green. If the marbles are stirred and mixed up so the different colored marbles are evenly distributed, it makes sense mathematically that if you pull out several handfuls of marbles, the proportion of red to blue to green will be the same as in the full barrel. You don’t have to count every marble in the barrel to show that 20 percent of the marbles are red; 50 percent are blue; and 30 percent are green. With a random sample, you can predict those percentages accurately.
However, if the different colors are not evenly distributed, you can pull out several handfuls from the barrel and you won’t get the right proportions. Your sample will be totally inaccurate if the blue marbles are all in one corner and the green are all at the bottom.
The different marble colors, in polling, represent different demographics of respondents. If a pollster gets the demographics right, survey research will be accurate. Thus, the number of conservatives, liberals, males, females, ages, religion, location, income, etc., in the sample surveyed must reflect the general population being surveyed. A randomly-drawn sample of 600 respondents, evenly distributed demographically, will accurately reflect the opinions of the general population.
Dan Jones is accurate because he works extremely hard to get the demographics and sampling correct.
Another important variable in political polling is who will actually turn out to vote. That’s a variable many pollsters have trouble with. A pollster can accurately measure support of politicians among the general public. But that doesn’t necessarily show who will win, because only the most motivated citizens may turn out to vote.
Thus, in conducting campaign research, pollsters must have techniques to screen out those who are not likely to vote.
At best, survey research is only a snapshot in time. Big events can rapidly change public opinion. Research taken weeks ago may not reflect opinions today.
Why do we conduct survey research? We believe it’s important for regular citizens and political leaders to understand how the population feels about important issues. However, I don’t believe politicians should make their public policy decisions based solely on what polling shows. What’s right and wrong, what one’s gut instinct says, what advice one receives, are as important, or more important, than knowing what the public thinks. But public opinion is still an important data point.
It’s worthwhile for news organizations to conduct independent survey research as a check on special interest research and polling done by candidates themselves. Candidates and interest groups regularly conduct polling, and will release results if it suits their strategy. It’s good to have independent research to keep them honest.
