Why polls on Trump Executive Orders give different results

David ThorntonFebruary 10, 2017

0 218 4 minutes read

There has been much celebrating in Republican circles over the recent poll results showing that slim majorities approve of President Trump’s Executive Orders including the controversial temporary immigration ban. One poll doesn’t represent the full picture, however, and other recent polls show that the waters of public opinion are sufficiently muddy to keep either side from claiming the support of a majority of the public.

The most recent poll, from Morning Consult/Politico, shows public opinion in favor of all of Trump’s Executive Orders. The immigration ban ranks as one of the most popular with 55 percent approval to 38 percent disapproval.

Interestingly, one day earlier, Politico headlined an article about another poll “Majority of U.S. voters oppose Trump immigration actions.” The Politico piece detailed a Quinnipiac poll that showed that voters disapproved of the immigration ban. Quinnipiac broke the Executive Order into parts and found that voters opposed the 90-day ban on immigrants from the seven listed nations by 51 to 46 percent. They also opposed the 120-day suspension of the refugee program by 60-37 percent and the indefinite ban on Syrian refugees by 70-26 percent.

CNN released a poll last week that also showed that public opinion was not on President Trump’s side. The CNN/ORC poll found that 53 percent opposed the travel ban while 47 percent were in favor.

Will this presidential election be the most important in American history?

Yes

All three polls also give an approval rating for President Trump. Morning Consult puts the president’s approval highest at 47 percent. Quinnipiac rates him lowest at 42 percent. CNN/ORC is in the middle at 44 percent.

So who is right when polls report different results? The tendency for many people is to simply pick the result that we like best, use it reinforce our preconceived ideas about what the rest of the country thinks and move on. President Trump, who recently tweeted that “any negative polls are fake news,” is a prime example of this mentality.

Some indicators of accuracy can be found by looking at the nuts and bolts of the polls. Often, a link to the actual poll can be found in news articles. If not, a web search can sometimes turn up the raw data that is available to journalists. I have linked directly to the raw polling data for the three polls discussed above.

The first indication of accuracy is who was polled. “Likely voters” are the most prized sort of sample group in polls. These are considered to be people engaged and informed enough to vote and who plan to do so. “Registered voters” are next best thing. These are people who probably have some awareness of current events and who have probably voted at some time in the past. From there, poll results get less accurate as we go to “adults” and “Americans.” These samples don’t necessarily screen for the ability to vote or interest in doing so.

Polling data will also give the sample size. Obviously, a larger sample is better and generates a smaller margin of error.

Often, polls will describe how the poll was conducted as well. This could be in person, on the telephone or on the internet. The methodology can affect the results. Politico notes that Trump’s numbers are better in online surveys than in polls conducted by phone. If the poll is a straw poll conducted on the internet by anyone who can get to the website, it is worthless for determining true public opinion.

So how do the samples in our recent polls stack up? The Morning Consult poll was conducted online among 2,070 registered voters. This generated a margin of error of plus or minus two percentage points. The Quinnipiac poll used landline and mobile phones to contact 1,155 voters. The smaller sample gives it a margin of error of 2.9 points. Finally, CNN/ORC interviewed 1,002 “adult Americans” by telephone. The margin of error on this poll is three points.

So all of the polls have pros and cons. The Morning Consult poll has the smallest margin of error, but the methodology may have inflated the pro-Trump score. Quinnipiac’s margin of error is larger, but telephone calls are more accurate, especially when mobile phones are included. CNN’s small size and sample that wasn’t restricted to voters is the least accurate.

Additionally, when examining polls, consider the way the question is asked. Is it asked in a misleading or biased way? Are there numerous possible answers or is it a binary choice? How many respondents chose “undecided” if that is an option?

Also consider the reputation of the pollster. Some polling firms are associated with a political party. For example, Public Policy Polling is a Democratic firm and their results may reflect a partisan bias. Rasmussen and Gallup are the gold standard of polling, but most polls that are associated with legitimate news sites are reasonably accurate. Morning Consult, Quinnipiac and CNN are all reputable pollsters.

It is worth noting that Rasmussen’s daily tracking poll shows Trump with 53 percent approval and 57 percent disapproval. Gallup, on the other hand, shows almost a mirror image with Trump’s approval at 43 percent and 52 percent disapproval.

Polling is hard.

So how should we interpret the varying results of polling on Trump’s Executive Orders? The best solution is to not put our eggs in one basket with any one poll. Consider each poll a snapshot of a landscape that is changing. The real answer is most likely somewhere near the average of the results.

Don’t just look at the raw numbers of the polls, especially if the differences are close, as there are. Instead, look for trends and the general direction of the movement of the numbers. Are things looking better for Trump, or worse? Is his margin increasing or narrowing? Discard outliers, polls that are dramatically different and isolated in their results, unless there is good reason to believe that trends are changing rapidly.

The most important takeaway on the current batch of polls is that the nation is split down the middle on the Executive Orders. Whether there is a slight majority in favor of President Trump’s actions or a slight majority against him, polling definitely shows that the president and the Republicans face opposition from a large segment of the electorate.

Originally published on The Resurgent