Basic statistical concepts - validity

Definition

Validity describes whether a measurement or study result actually captures the true value of what it is intended to measure - reflecting the absence of systematic error/bias, distinct from precision (reproducibility), which reflects the absence of random error
A foundational measurement quality concept relevant to critically appraising any clinical measurement, diagnostic test, or study finding

Systematic bias (a consistent, directional error - e.g. a miscalibrated instrument, a study population that is not representative of the intended target population, or a flawed measurement method) shifts results away from the true value in a consistent direction, unlike random error, which scatters results around the true value without a consistent directional pull - this distinction explains why increasing sample size (which reduces random error's impact) does not correct for systematic bias, no matter how large the sample

Assessing validity in a study requires examining the study design and methodology for potential sources of systematic bias (selection bias, measurement bias, confounding) rather than looking at precision/sample size alone
Internal validity (whether the study result accurately reflects what happened within the study population) and external validity (whether the result generalises to other populations/settings) are related but distinct validity concepts
Internal vs external validity distinction, selection bias and confounding as validity threats, large sample size not correcting for systematic bias, precision vs validity distinction (see )

Larger sample size alone does not fix a validity/bias problem - a very large, very precise (narrow confidence interval) study can still produce a systematically biased, invalid result if the underlying methodology has a flaw
Critical appraisal of study methodology (randomisation, blinding, representative sampling) is the appropriate way to assess and address validity concerns, not simply increasing sample size

Increasing sample size improves precision (narrower confidence intervals) but does nothing to correct systematic bias/validity problems - a very large study with a flawed, biased methodology remains invalid regardless of how precise (narrow) its confidence interval appears, which is why critical appraisal of study methodology, not just study size, is essential when interpreting results

4 more sections, plus exam facts

Premium unlocks every note across every specialty, and the full exam fact library behind it.