“What’s up with your new approval rating polls? They seem lower than most past polls,” is a common refrain we’ve heard from several Utah politicos since we unveiled the first batch of results in July.
Politicians like Gov. Gary Herbert, who are used to polls with job approval ratings in the mid to upper 60s are now suddenly at 48% approval, which is a dramatic drop.
The temptation is to discount our polling as an outlier, but that would be a mistake.
The reason for the decline in approval rating is a fundamental shift in the way we asked the question. Instead of giving respondents the ability to only“approve” or “disapprove” of an officeholder’s job performance, we added another choice to the matrix: “Neither approve nor disapprove.” That neutral middle selection is the reason the numbers are lower.
Let’s turn to UtahPolicy.com pollster Kelly Patterson of Y2 Analytics, who explains why we decided to include the neutral selection in our surveys.
“There is actually a large literature, as you might expect, on whether or not to include a midpoint in a question scale. The debate hinges on what you think is going on in the respondent’s mind and then whether the scale you provide allows them to accurately map that on to the scale. Many argue that you need a midpoint in order to help sift out those individuals who have nonattitudes. Furthermore, if you are really interested in intensity of opinions, then you need to offer a midpoint or at least a branching scheme where you ask people the direction of their opinion and then follow that up with an intensity opinion. If you theorize that people genuinely may neither approve or disapprove, then you should include a midpoint. My guess is that pollsters in this state have long simply ignored that debate or simply misconceptualized approval. We have measured approval both ways over the years.”
For example, we found 13% of Utahns said they neither approved nor disapproved of Gov. Herbert, while 23% felt that way about Lt. Gov. Cox. In our congressional job approval ratings, the neutral choice got between 10 and 20% of respondents, depending on the subject of the survey. Only 4% of Utahns say they felt neutral about how Pres. Donald Trump is doing his job.
So, does adding a middle choice make results more accurate? Here’s Patterson again.
“As you may have noticed, there are tradeoffs. We could easily argue that the question with the midpoint creates more variation( a good thing) and actually allows for individuals to map their cognitive world on to the scale (also a good thing). The downside is that a midpoint allows people to satisfice (they are lazy and simply pick the easy way out). If you think people really do have approval opinions about politicians, then you make the argument that you need to force them to choose. It seems on balance that a midpoint is a better option, but it is not a slam dunk. Those people who dismiss our numbers are making implicit or explicit assumptions about psychology of which they may not be aware. Not a totally new development given the fact that most people do not spend time in the world of the psychology of the survey response.”
UtahPolicy.com and Y2 Analytics have teamed up on the Utah Political Trends survey project, a monthly poll measuring the attitudes of Utah voters.
