The reports of another supposed breakthrough in genetic research are, like so many before it, rather exaggerated. Last week, a New Scientist editorial commented that neuroscience
” is plagued by false positives and other problems. … Scientists are under immense pressure to make discoveries, so negative findings often go unreported, experiments are rarely replicated and data is often “tortured until it confesses”. … Genetics went through a similar “crisis” about a decade ago and has since matured into one of the most reliable sciences of all. “
Yesterday the newspapers were stuffed with reports from that most reliable and mature of sciences, concerning the discovery of 11 genes newly implicated in the causation of Alzheimers. This is from the Independent:
The role of the immune system in defending the brain against Alzheimer’s disease has been revealed in a study identifying 11 new genes that could help to trigger the most common form of senile dementia.
There’s more than enough there to be able to tell that the report is confused. In the first place, Alzheimer’s disease is not a single disease entity; it’s a syndrome. The term is used as a residual category for any form of dementia where there isn’t as yet a clear understanding of the process. Over the years, the size of that residuum has gradually been reduced as various specific disease entities have been identified – Pick’s, Huntington’s, Parkinsonian dementia, Lewy body, CJD and so on. The process of refinement still has a long way to go. Second, there is no evidence that Alzheimer’s is genetically determined or ‘triggered’ by particular genes. The study does not actually claim to show that the immune system defends against Alzheimer’s. All it does it to identify a group of SNPs or snips (single nucleotide polymorphisms to their friends) associated with the immune system which show some association with the diagnosis of dementia. That’s an interesting finding, because it suggests that it may be worthwhile to examine immune systems to see what connections emerge. It’s not the same thing as showing that genes cause Alzheimer’s.
However, it’s not possible to exonerate the authors of the paper altogether of blame for the misrepresentation. The title of the article, published in Nature Genetics, is: “Meta-analysis of 74,046 individuals identifies 11 new susceptibility loci for Alzheimer’s disease”. This does assume that the associations show ‘susceptibility loci’, and it emphasises that it’s a big study, which implies that it has greater authority as a result. The conclusion suggests that what needs investigating is the potential association with the risk of Alzheimer’s.
There are three common errors here: the paper commits some of the cardinal sins of statistics.
- Confusing association with causation. An association doesn’t in itself tell us what the influence of genes is or what the direction of causation is. It follows that assocation with certain genes doesn’t reveal susceptibility.
- Confusing significance with risk factors. A relationship can be highly statistically significant although its effects are very limited. (On a graph, it’s the slope of the regression line that really matters rather than the closeness of fit of the observations). It’s possible that some small part of the response is attributable to the associated factor, and in medical terms that’s potentially important – it could relate to a particular condition – but that’s not equivalent to a risk factor, and in any case the work done doesn’t identify that.
- Fishing, or data mining. In any very large body of data, there will be some unusual associations – it’s in the nature of the exercise. It doesn’t follow that those associations can be invested with meaning. This study fishes for the data in a massive pool – over 17,000 people with Alzheimer’s, over 37,000 controls and more than 7 million SNPs. Then in stage 2 there were 8572 people with dementia, 11,312 controls and 11,632 SNPs. The significance levels were strict (p < 5 per 10*-8), but the sheer size of the data sample makes the statistics more problematic, not less so. The method can’t do more than suggest that some patterns merit further investigation.
See also: http://blog.spicker.uk/dementia-we-shouldnt-expect-miracles-from-drug-cures/