Peer Review fails again

I have written earlier on the connexion between vaccines and autism, expressing my desire to see more research before reaching a conclusion. Just seeing research, however, is not sufficient; it must be evaluated. This is supposedly the task of Peer Review, which is thought to ensure that no pseudoscience makes it into print in the scholarly journals. Note that Richard Horton, editor of The Lancet, wrote this about the peer review process (but not in his own magazine!):

We know that the system of peer review is biased, unjust, unaccountable, incomplete, easily fixed, often insulting, usually ignorant, occasionally foolish, and frequently wrong.

Perhaps nowhere has this process been demonstrated to have utterly failed more than in the case of Andrew Wakefield, who has been ostracized from the medical community over the publication (in The Lancet) of his study of the link between autism and the MMR vaccine. The first set of allegations against him (half of which were dismissed as unfounded), and his response, can be seen at this link. The 143-page fact sheet from the preliminary license revocation hearing of 3 of the article's authors (released this past January after 2½ years of investigation) can be found here.

As nearly as I can figure out, the only piece of evidence collected by Dr. Wakefield's team that is actually disputed is the condition of the children's intestines, and--what do you know--the original records have disappeared. This is how the Lancet article reads:

Findings

Onset of behavioural symptoms was associated, by the parents, with measles, mumps, and rubella vaccination in eight of the 12 children, with measles infection in one child, and otitis media in another. All 12 children had intestinal abnormalities, ranging from lymphoid nodular hyperplasia to aphthoid ulceration. Histology showed patchy chronic inflammation in the colon in 11 children and reactive ileal lymphoid hyperplasia in seven, but no granulomas. Behavioural disorders included autism (nine), disintegrative psychosis (one), and possible postviral or vaccinal encephalitis (two). There were nofocal neurological abnormalities and MRI and EEG tests were normal. Abnormal laboratory results were significantly raised urinary methylmalonic acid compared with age-matched controls (p=0.003), low haemoglobin in four children, and a low serum IgA in four children.

The article concludes:

We did not prove an association between measles, mumps, and rubella vaccine and the syndrome described. Virological studies are underway that may help to resolve this issue. If there is a causal link between measles, mumps, and rubella vaccine and this syndrome, a rising incidence might be anticipated after the introduction of this vaccine in the UK in 1988. Published evidence is inadequate to show whether there is a change in incidence or a link with measles, mumps, and rubella vaccine. A genetic predisposition to autistic-spectrum disorders is suggested by over-representation in boys and a greater concordance rate in monozygotic than in dizygotic twins. In the context of susceptibility to infection, a genetic association with autism, linked to a null allele of the complement (C)4B gene located in the class III region of the major-histocompatibility complex, has been recorded by Warren and colleagues. C4B-gene products are crucial for the activation of the complement pathway and protection against infection: individuals inheriting one or two C4B null alleles may not handle certain viruses appropriately, possibly including attenuated strains. Urinary methylmalonic-acid concentrations were raised in most of the children, a finding indicative of a functional vitamin B12 deficiency. Although vitamin B12concentrations were normal, serum B12 is not a good measure of functional B12 status. Urinary methylmalonic-acid excretion is increased in disorders such as Crohn’s disease, in which cobalamin excreted in bile is not reabsorbed. A similar problem may have occurred in the children in our study. Vitamin B12 is essential for myelinogenesis in the developing central nervous system, a process that is not complete until around the age of 10 years. B12 deficiency may, therefore, be a contributory factor in the developmental regression. We have identified a chronic enterocolitis in children that may be related to neuropsychiatric dysfunction. Inmost cases, onset of symptoms was after measles, mumps, and rubella immunisation. Further investigations are needed to examine this syndrome and its possible relation to this vaccine.

One thing that greatly complicates the discussion is the latitude with which a diagnosis of Autism is dispensed. I read a billboard just the other day, that every 20 minutes another child is diagnosed with autism (where--in the hospital emergency room?). Yet the same child, with the same symptoms, can be diagnosed either with autism or pervasive development disorder, just on the basis of either not wanting to hurt the parents' feelings on the one hand, or qualifying for taxpayer-funded intervention on the other. It has become a political diagnosis as much as a medical one.

The research I have seen definitely points in the direction of a 3-fold link between:

1) Reaction to the MMR shots
2) Verbal and social regression around age 18 months
3) Casein and gluten allergy with cerebral symptoms

But millions of children being diagnosed with autism spectrum disorder don't necessarily show this triad of symptoms. If meaningful research is to be done, children that show this triad must first be segregated from the great mass of those labeled with ASD.

In the mean time, informed parents will continue to do their own research and work out their own treatment regimens. Classic autism is clearly both preventable and treatable. If existing treatments don't work in any given case, it only shows that more study needs to be done.

What is medicine's 5 sigma? by Richard Horton, Published in "The Lancet" 11 April 2015

The following excerpt is taken from a magazine we've covered before, written by the editor himself. I'd link to the page, but I don't expect it to stay up for long.

 “A lot of what is published is incorrect.” I’m not allowed to say who made this remark because we were asked to observe Chatham House rules. We were also asked not to take photographs of slides. Those who worked for government agencies pleaded that their comments especially remain unquoted, since the forthcoming UK election meant they were living in “purdah”—a chilling state where severe restrictions on freedom of speech are placed on anyone on the government’s payroll. Why the paranoid concern for secrecy and non-attribution? Because this symposium—on the reproducibility and reliability of biomedical research, held at the Wellcome Trust in London last week—touched on one of the most sensitive issues in science today: the idea that something has gone fundamentally wrong with one of our greatest human creations. * The case against science is straightforward: much of the scientific literature, perhaps half, may simply be untrue. Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has taken a turn towards darkness. As one participant put it, “poor methods get results”. The Academy of Medical Sciences, Medical Research Council, and Biotechnology and Biological Sciences Research Council have now put their reputational weight behind an investigation into these questionable research practices. The apparent endemicity of bad research behaviour is alarming. In their quest for telling a compelling story, scientists too often sculpt data to fi t their preferred theory of the world. Or they retrofi t hypotheses to fi t their data. Journal editors deserve their fair share of criticism too. We aid and abet the worst behaviours. Our acquiescence to the impact factor fuels an unhealthy competition to win a place in a select few journals. Our love of “significance” pollutes the literature with many a statistical fairy-tale. We reject important confirmations. Journals are not the only miscreants. Universities are in a perpetual struggle for money and talent, endpoints that foster reductive metrics, such as high-impact publication. National assessment procedures, such as the Research Excellence Framework, incentivise bad practices. And individual scientists, including their most senior leaders, do little to alter a research culture that occasionally veers close to misconduct. * Can bad scientific practices be fixed? Part of the problem is that no-one is incentivised to be right. Instead, scientists are incentivised to be productive and innovative. Would a Hippocratic Oath for science help? Certainly don’t add more layers of research red tape. Instead of changing incentives, perhaps one could remove incentives altogether. Or insist on replicability statements in grant applications and research papers. Or emphasise collaboration, not competition. Or insist on preregistration of protocols. Or reward better pre and post publication peer review. Or improve research training and mentorship. Or implement the recommendations from our Series on increasing research value, published last year. One of the most convincing proposals came from outside the biomedical community. Tony Weidberg is a Professor of Particle Physics at Oxford. Following several high-profi le errors, the particle physics community now invests great eff ort into intensive checking and rechecking of data prior to publication. By fi ltering results through independent working groups, physicists are encouraged to criticise. Good criticism is rewarded. The goal is a reliable result, and the incentives for scientists are aligned around this goal. Weidberg worried we set the bar for results in biomedicine far too low. In particle physics, significance is set at 5 sigma—a p value of 3 × 10–7 or 1 in 3·5 million (if the result is not true, this is the probability that the data would have been as extreme as they are). The conclusion of the symposium was that something must be done. Indeed, all seemed to agree that it was within our power to do that something. But as to precisely what to do or how to do it, there were no firm answers. Those who have the power to act seem to think somebody else should act first. And every positive action (eg, funding well-powered replications) has a counterargument (science will become less creative). The good news is that science is beginning to take some of its worst failings very seriously. The bad news is that nobody is ready to take the first step to clean up the system. Richard Horton richard.h orton@lancet.com

Pushing an Abortion Drug on Third World mothers

NPR's Morning Edition ran an interesting story this morning. Quoting a new study just published in The Lancet, Brenda Wilson reports that "a drug originally developed to treat ulcers," Misoprostol, could "save thousands of lives of women in developing countries" by controlling post-partum bleeding--a condition, by the way, presently treated with Pitocin, a synthetic oxytocin. The study, conducted in India, was funded by the Global Network for Women's and Children's Health research, a partnership between the US National Institute of Child Health and Human Development and the Bill and Melinda Gates Foundation.

What the three-minute report is careful not to say is that while Misoprostol was originally developed as an ulcer treatment, its primary use for the last decade has been as an abortifacent--one that, as it turns out, often causes extensive bleeding when administered to pregnant women. Yes, that's right--Misoprostol is a constituent drug of the medical abortion procedure.

NPR's Wilson predicts that the extremely low cost and user-friendly nature of Misoprostol will result in it being used by uncertified birth attendants throughout the Third World. Unspoken is the implication that it could make abortion as easy a matter as a trip to the nearest pharmacy.

This is news. Too bad it's not being reported.