Information Valuation

Information valuation is a key challenge. Having a skeptical mind makes it difficult to have confidence in nearly any report. I’m sure that I’m not unique in adding a “suspicions confirmed” filter to incoming information.

NIST scientists recently pointed out that that the term “likelihood ratio” is being misused in many forensic situations. The problem is that the likelihood values are too subjective. Compounding them makes estimates even wilder. (See http://americanlaboratory.com/Blog/343374-Lack-of-Rigor-and-Reproducibility-Raises-Question-About-Likelihood-Ratios/.)

Another problem arises when evaluating risks associated with data mining. Large databases are being compiled and then examined for correlations that might be useful in guiding diet and healthcare. In “A Link Between Alcohol and Cancer? It’s Not Nearly as Scary as It Seems,” Professor Aaron E. Carroll reports that heavy alcohol consumption is attributed to 3.5% of American cancer deaths. Light drinking had a relative risk of 1.04%. A 40-year-old female has an absolute risk of 1.45% of developing breast cancer in the next decade. Adding on the relative risk for light drinking increases her risk by 1.45 × 1.04 = 1.51%. This is a very small risk. Plus, this prediction misses the protective effects of light drinking. Carroll points out that the 2013 study in Annals of Oncology involving 23 cancers showed that light drinking was protective, moderate drinking was neutral, and heavy drinking was indeed detrimental.

According to Carroll, cherry-picking data in large studies can find large percentage correlations in rare events. These headline-grabbing correlations often miss protective factors such as the reduction of cardiac risks that can arise from reduced stress. Carroll advises: 1) Focus on large absolute risks. A 30% increase from 1 to 1.3 is not significant, but a 30% increase in a 30% absolute risk to 39% is more concerning. 2) Do not give much weight to correlations from data mining unless the causal relationships have been established. 3) Do not focus on one disease or peril while neglecting others. 4) Do not cherry-pick data but develop a holistic picture. 5) Acknowledge the perils and benefits. “Consider the cost and joy.”

Carroll says that following these rules will avoid getting caught up in sensational headlines, which will probably lead to happier and probably healthier lives.

As chemists, we are exposed to a range of perils that are not typical of the population at large. We must act prudently by keeping an open mind to the absolute risks that we encounter in our labs and private lives. Our training should help us evaluate potential threats and opportunities for mitigation. We can lead the public by helping them see the value of focusing on the clear dangers and avoiding fears generated by sensational headlines.

Robert L. Stevenson, Ph.D., is Editor Emeritus, American Laboratory/Labcompare; email: [email protected].

Related Products

Comments