Election surveys: Science or séance?

BY ANTONIO CONTRERAS         FEBRUARY 07, 2019

THIS is now the season of election surveys and once again people are asking if it is indeed possible for the opinion of millions of voters to be validly revealed by a survey that asked only 1,800 individuals. Is there a science behind surveys, or is this just like the Delphian oracle, or some kind of a glorified séance where survey takers act like spiritualists receiving information from the dead?

At the outset, it must be emphasized that there is a science behind surveys, but it should also not be denied that this science can be undermined when surveys are gamed and weaponized as a political tool.

The reliability of surveys rests on the existence of what is referred to as a properly randomized sample. This is the key why 1,800 respondents would be enough to measure the opinion of the entire voting population. A random sample ensures that every member of the population has an equal chance of being chosen as part of the sample. Sampling may be too abstract for a layperson to understand. The best way to appreciate it is to know that product quality control is done by randomly checking only a sample of products, whether it be light bulbs or soft drinks or vehicles. Not every bottle of milk must be opened to check for quality. Only a sample of bottles need to be examined and tested, as long as they are chosen randomly, and without manifest bias.

This then begs the question that unlike products that are somewhat much alike, since they are all milk, or light bulbs, or cars, applying the principle on people may be problematic considering that human beings are more diverse. However, this doubt can be dispelled by the fact that if the sample is truly randomly selected, it will then reflect the diversity of the entire population, and its composition will truly represent the various classes and categories. There are also ways of randomization that can further enhance representativeness of the entire population, such as drawing the sample proportional to the distribution of some predetermined social variable, such as gender or social class, whichever is deemed appropriate and significant to the issue for which the survey is being conducted.

Ideally, a sample is usually drawn by choosing randomly from a list. Unfortunately, in the case of election surveys, we are talking of a population by the millions. This is where the task becomes challenging.

In situations where mobile phone penetration is high, the sample can be drawn by generating random phone numbers. Here, the manner of conducting the survey will then be not face to face, but by phone by simply dialing the randomly chosen numbers. Appropriate replacement methods are employed in anticipation of unanswered calls, or people refusing to participate in the survey. However, this may not produce a reliable and representative sample when applied in the Philippines, since it effectively excludes people without mobile phones. This then violates the principle of randomization where every member of the population must have an equal and fair chance of being chosen.

In the end, the usual practice in the Philippines is to adopt a stratified random sampling, where the unit being sampled are individual voters drawn from randomized locations such as cities/municipalities, barangays, households and voting precincts. Here, the sampled 1,800 voters are distributed proportionally to the different regions, after which the sampling points in each region are drawn by randomly choosing first the sample municipalities/cities, then randomly drawing from each municipality/city the sample barangays, then randomly drawing from each barangay either the sample household, or the sample precincts.

If the lowest sampling unit is a sample precinct, it is just a matter of drawing randomly from the registered voter’s list of that precinct. Ideally, this is the best method that can approximate perfect randomization since if done properly, this ensures that every voter has an equal chance of being drawn as a sample considering that all geographical units, from municipalities/cities to barangays to precincts were all given equal chance to be chosen, and that the number of samples per region is proportional to the voting population of that region.

However, when the lowest sampling unit is a household, then some bias might enter the process in the identification of who in the household to interview. If the method chosen is the head of the household, then certainly this violates the basic principle of a truly random sample since it now then effectively eliminates other members from being part of the sample. One other method is simply to consider the first person who answers the door who is also an eligible voter. While this is an improvement, it nevertheless has a bias against people who may be working, or who are regularly away from home.

One method that is also used that can magnify the bias, and therefore undermine the reliability and validity of the survey, is when the lowest sampling unit is drawn not from a voters list from a randomly chosen precinct in a sampled barangay, but a house that is identified not by randomization, but spatially from a landmark like a flagpole or a barangay center in the sample barangay. In this method, the surveyor, using some heuristic guide, would draw the sample respondent relative to the landmark, by setting a distance and a direction, and then drawing the sample household from that fixed location relative to the landmark. This can be the first individual who is a qualified voter who answers the door, or the head of the household of the house located in that spot. This method is extremely problematic considering the bias it holds in favor of those living near landmarks, even as it carries with it the inherent structural biases that are associated when the individuals are not drawn from a voters list.

Dissatisfied supporters of candidates have accused surveys of being biased, and of being designed only to condition the minds of people to create a bandwagon effect, or of being blatant partisan strategies to manipulate voter preferences. It is therefore incumbent upon survey organizations like SWS, Pulse Asia and Publicus, among others, to also help in educating the public about the science of surveys. And they can do this by being transparent, and by publishing in detail every time they release a survey the location of their samples. It is not enough to simply publish the number of respondents from the four geographical locations of NCR, balance Luzon, Visayas and Mindanao. In addition, they should also indicate the detailed methodology of how they eventually chose their sampled respondents. It is only through these that they can dispel the doubts of the voters, and make them realize that they are scientists communicating the results of a statistically reliable and valid scientific exercise, and not spiritualists who communicate the results of their séance with the dead and the imagined.

https://www.manilatimes.net/election-surveys-science-or-seance/507847/?utm_source=jubna&utm_medium=jubna_widget&utm_campaign=jubna_trending&utm_term=jubna_recommendation