How Science Works

How Science Works

Science is the most reliable method humans have developed for separating what is true from what we want to be true. But the way science is practiced, communicated, and consumed creates systematic distortions that allow false or overstated claims to proliferate — especially through media coverage.

The Scientific Method

The scientific process: observe a phenomenon, formulate a hypothesis (a testable prediction), design an experiment that could prove the hypothesis wrong, collect data, analyze results, and draw conclusions. The critical feature is falsifiability — a genuine scientific hypothesis must be capable of being disproven. Claims that cannot be tested or falsified are philosophy or theology, not science.

A single experiment proves nothing. Scientific knowledge accumulates through replication — the same finding appearing across multiple studies, using different methods, conducted by different researchers with no relationship to the original team.

How Research Gets Distorted

Publication bias: Scientific journals prefer publishing positive results ("this drug works") over null results ("this drug did nothing"). This creates a skewed record — the published literature overrepresents findings that support hypotheses. Meta-analyses that pool all studies on a topic are contaminated by this bias.

P-hacking (data dredging): Researchers sometimes analyze data multiple ways, across multiple subgroups, until something reaches p < 0.05 — then report only that result as if it was the planned analysis. If you run 20 comparisons on random data, you are statistically likely to find one that appears "significant" by chance alone.

Small sample sizes: Many studies, especially in psychology, are conducted on small groups of university students. A study finding a "significant" effect in 40 participants may have no ability to detect whether the effect exists in the general population. Small samples produce unreliable estimates with wide uncertainty.

HARKing (Hypothesizing After Results are Known): Presenting a hypothesis as if it was pre-specified before the study, when it was actually generated after looking at the data. This violates fundamental scientific logic but is difficult to detect from a published paper.

How Headlines Lie About Studies

Media coverage of scientific research is almost universally distorted in predictable ways:

Correlation reported as causation: "People who eat chocolate are thinner." The study showed a correlation — both things are true of the same people — not that chocolate causes thinness. Confounding variables (wealthy people eat better chocolate and also exercise more) typically explain correlations between lifestyle factors and health outcomes.

Effect sizes are omitted: A drug that reduces heart attack risk by 30% sounds impressive. If the baseline risk is 1% (absolute risk goes from 1% to 0.7%), the drug reduced your actual risk by 0.3 percentage points. Headlines report the relative risk reduction (30%) not the absolute risk reduction (0.3%) because it sounds more dramatic.

Single studies reported as established fact: "New study finds..." is the journalist's way of saying "preliminary research with no replication suggests..." One study, regardless of its quality, cannot establish a scientific fact.

Evaluating Research Yourself

Questions to ask before accepting a study's conclusions:

1. What was the sample size? Under 100 participants is low for most questions. Under 1,000 for population-level health questions is cause for skepticism.
2. Was it randomized and controlled? Observational studies (watching what people do) cannot establish causation. Randomized controlled trials (randomly assigning people to treatment and control groups) can.
3. Has it been replicated? One study proves little. Multiple replications by independent groups prove much more.
4. Who funded it? Industry-funded studies are more likely to find results favorable to the funder's product. This does not make them wrong, but it is a reason to look for independent replications.
5. What do the actual numbers show? Find the original study and read the numbers, not just the conclusion.

Scientific literacy is not about distrusting science — it is about understanding that the scientific process is human, imperfect, and self-correcting over time.