It was enjoyable last week to listen to pollster Dan Jones, the old survey research master, talk about the state of Idaho politics. He spoke in Boise at the Zions Bank Premier Wealth Management Speaker Series luncheon before many Idaho business and political leaders.
The business of survey research is worth discussing because it is an industry in the midst of dramatic change, influenced, like so many other industries, by rapid technological advances. There have been a number high-profile examples of inaccurate campaign surveys that didn’t match election outcomes.
Survey research remains important to the public policy industry but, as Jones noted, it is becoming more difficult to conduct accurately. It’s harder to get an accurate scientific sample when fewer people have landlines, and cell phone numbers are difficult to obtain. In telephone surveys, good pollsters use a combination of landline and cell phone interviews, plus on-line panels of respondents.
Jones works extremely hard to obtain good samples. Without a sample that reflects the population demographics, a survey is likely to be inaccurate.
Wise candidates and political practitioners pay attention to survey research, but they don’t live by it. A poll reflects a snapshot in time, and opinion can change quickly. After a highly-controversial or emotional event, opinion on can spike dramatically before returning to normal.
Some people are critical of news organizations that conduct polls, accusing them of trying to influence public opinion. But by measuring public opinion, we’re not necessarily trying to influence it. We’re providing one more data point leaders can use to make decisions. It’s helpful to know how citizens feel, but that obviously shouldn’t be the only thing policymakers consider. Sometimes public sentiment is simply wrong. The idea that every decision a politician makes should reflect majority opinion is wrong.
Policymakers should use survey information as one more data source—but that’s all. One more relevant bit of intelligence in the mix. Important decisions should be based on all available information, including polling information, advisor and expert opinion, and one’s principles, beliefs, experience and intuition.
Sometimes having good in-depth survey research on a topic can help policymakers know what messages to use to influence public opinion. The education process is critical to good policymaking, and having good research can be very helpful.
In a campaign, survey research can obviously be crucial. I recall one campaign in waning days of the primary election. Suddenly, we received a spate of calls about vitriolic flyers attacking our candidate being passed out in shopping malls. We also became aware of a phone bank campaign making outlandish claims about our candidate. The campaign staff became very worried, trying to figure out how to respond, whether to call a press conference, whether to issue a press release.
But we were doing daily tracking surveys at the time, and we decided to wait and see what the next day’s tracking results looked like. We didn’t see a blip, no erosion of support. The following day, despite worried calls from supporters, we still didn’t see any deterioration in our overall support. We continued on with our campaign plan and won the primary. It would have been very easy to overreact and draw attention to the accusations if we didn’t have good research.
One of the dangers of campaign polling is relying on research that accurately measures overall public opinion — but not the opinion of those who are actually going to vote. The genius is in getting the right sample, people who are really going to vote.
Today, gigantic retailers and data firms mine Internet and on-line consumer movements and habits to gather data about people. They know a great deal about consumer preferences and behavior, and they merge the data they gather with Census data and other transactional data. Smart political wonks are trying to apply those capabilities and practices to politics and will, no doubt, one day be successful.
But for now, using an old-fashioned pollster like Dan Jones is still the best way to measure citizen opinion on political topics.
