ASL Evidence Meter

Sound bytes can be a dangerous thing. And research conclusions with their many numbers of probabilities have little value without context but so often make for great sounds bytes which generate much hype and noise.

We suggest asking the 5 questions below when reading about or listening to any claims about the scientific evidence.

This is not exhaustive but does provide an initial sanity check and framework for identifying more context.

Button

Q 1. What type of research are we looking at? '

Animal? Human? Has it been replicated in Humans? Bio markers or actual health outcomes data? Meta-analysis of multiple studies?

Q2. What was the study time frame?

6 weeks? 6 months? A year? Longitudinal (many years)? Is this something that needs more time to assess mid or long term impact?

Q3. Does the study assess indicators for risk or actual measurable outcomes in humans?

Both have value, but actual measured outcomes is what I always hope to see.

Q4. What else could be influencing the significance between control and intervention groups? What is actually being compared and/or controlled for?

Example 1. Background diets and lifestyles habits can bias results if not controlled for when changing one food item or behavior.

Example 2. Are threshold factors at play? If applicable to the intervention being assessed, one may not see any significant results until the threshold is actually crossed - over or under. While the conclusion may be true, does it paint the full and true picture about the pros and cons for a specific behavior if the discussion about threshold is not addressed (when applicable)? (Here most of us need to rely on analysis by the experts who are aware of these threshold considerations!)

Q5. How meaningful are the "significant differences" quoted? Numbers are easily sensationalized.

Example: Which is greater? A 100% increase of a 1% increased risk of cancer when your current risk is 1% or a 17% probability of acquiring Alzheimer's because your have one APOE 4 allele.

Answer: 100% (two fold) increase of 1% is 2%. a 17% probability of acquiring Alzheimer's is 17%, more than 8 times the probability of the 2% cancer risk.

Everyone has their own sensitivity to probability levels. The important take-away here is to ensure you are reading the numbers in context so you make informed choices! Headlines will rarely make this distinction for you.

Q6. What questions were raised by the research?

Most research triggers more questions to explore to better understand the findings and what else may be at play. A lack of new questions makes me wonder what is not being shared.