Ten flaws of surveys

Ten flaws of surveys | Daily Tribune

If poll surveys capture reality with absolute certainty, there’ll be no need for elections. Precisely because of their failings, nay flaws, polls could only be of any help if predictions come out “picture perfect” on election day, except that they prove to be more a case of “garbage in, garbage out.”

In a recent interview, fast-rising senatorial candidate, Rep. Rodante Marcoleta, expressed dismay over surveys without explaining further. Suppose we run down some possible explanations why the ranking order of senatorial candidates released by two dominant polling circuits appear to be few and far between?

Let us take the case of the survey findings released in September by these reputable pollsters marked Circuit X and Circuit Y.

This piece foregrounds certain flaws that may tend to undermine the sacred right of citizens to vote in an enabling environment free of interference — meaning of informed voters rather than misinformed by the apparent “deception” that surveys tend to project.

One, on their respective shortlists of senatorial candidates, one can cast serious doubts on their validity. Circuit X enrolled 40 prospects while Circuit Y had 74 and both had disparate variances in terms of the order of the top 12 or the winning circle.

Two, the names on their respective lists were entirely anecdotal, an uncharacteristic exercise in guesswork rather than evidence-based or unquestionably verifiable. The truth, however, is that their shortlists included names not found in the Commission on Elections’ own roster of senatorial candidates.

Three, if the polling circuits’ preliminary dataset are flawed, there’s no way their conclusions can be true. Or, why include names of those not running for senator (i.e. “Digong” Duterte, “Dick” Gordon, “Leni” Robredo, “Spox” Roque, “Gibo” Teodoro, “Frank” Drilon, Ralph Recto, “Chel” Diokno, Cesar Montano, “Mar” Roxas, “Bistek” Bautista, Vilma Santos-Recto, Korina Sanchez, Persida Acosta, Ted Failon and lesser known mortals), not to mention the unbelievable discrepancies in their ranking.

Four, why create the illusion that all those who had ranked had been vetted as senatorial candidates when the Comelec roster showed otherwise? Put another way, don’t the pollsters realize how their undue inclusion or exclusion could seriously distort the ranking outcome?

Five, how do these lapses affect the quality and accuracy of their surveys? How could they argue that their lists had been contemplated as the possible candidates for the Senate when prudence dictates that pollsters would be better off relying on vetted data.

Six, be that the case, the body of evidence is a naked assumption bereft of truth content. It fails all known tests of what survey data should be, let alone how it is scientifically arrived at — desirably neither anecdotal nor conjectural.

Seven, as basic as laid down datasets rest on false assumptions or unintelligently assumed possibilities, it would defy reason, logic, and evidence if pollsters insist to “scratch apple” from nothing. Statistics must not deal with mere “make-believe” or they risk presenting their findings akin to a mirage.

Eight, all the flaws or failings indicatively committed in the September survey releases on senatorial candidates immediately qualify as the proper subject of a congressional inquiry. The end in view of this legislative feat is noble insofar as it can result in tasking the proper agency of government to forthwith regulate private entities that forecast matters of extreme public interest.

Nine, perhaps, the authors of these “field surveys,” if we may call them that, insofar as they profess to present true, evidentiary, verifiable information of their statistical findings, must meet some stringent criteria laid down by a regulator. For instance, how in the first place were the respondents in the surveys selected?

Ten, were the survey instruments sent to a representative sample of the population? Their shortlists of senatorial candidates necessitate proof of expert judgment that assumptions were evidence-based; not purely inferential or plucked from the haze and maze of unexplainable probability.