What is medicine's 5 sigma? by Richard Horton, Published in "The Lancet" 11 April 2015

The following excerpt is taken from a magazine we've covered before, written by the editor himself. I'd link to the page, but I don't expect it to stay up for long.

 “A lot of what is published is incorrect.” I’m not allowed to say who made this remark because we were asked to observe Chatham House rules. We were also asked not to take photographs of slides. Those who worked for government agencies pleaded that their comments especially remain unquoted, since the forthcoming UK election meant they were living in “purdah”—a chilling state where severe restrictions on freedom of speech are placed on anyone on the government’s payroll. Why the paranoid concern for secrecy and non-attribution? Because this symposium—on the reproducibility and reliability of biomedical research, held at the Wellcome Trust in London last week—touched on one of the most sensitive issues in science today: the idea that something has gone fundamentally wrong with one of our greatest human creations. * The case against science is straightforward: much of the scientific literature, perhaps half, may simply be untrue. Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has taken a turn towards darkness. As one participant put it, “poor methods get results”. The Academy of Medical Sciences, Medical Research Council, and Biotechnology and Biological Sciences Research Council have now put their reputational weight behind an investigation into these questionable research practices. The apparent endemicity of bad research behaviour is alarming. In their quest for telling a compelling story, scientists too often sculpt data to fi t their preferred theory of the world. Or they retrofi t hypotheses to fi t their data. Journal editors deserve their fair share of criticism too. We aid and abet the worst behaviours. Our acquiescence to the impact factor fuels an unhealthy competition to win a place in a select few journals. Our love of “significance” pollutes the literature with many a statistical fairy-tale. We reject important confirmations. Journals are not the only miscreants. Universities are in a perpetual struggle for money and talent, endpoints that foster reductive metrics, such as high-impact publication. National assessment procedures, such as the Research Excellence Framework, incentivise bad practices. And individual scientists, including their most senior leaders, do little to alter a research culture that occasionally veers close to misconduct. * Can bad scientific practices be fixed? Part of the problem is that no-one is incentivised to be right. Instead, scientists are incentivised to be productive and innovative. Would a Hippocratic Oath for science help? Certainly don’t add more layers of research red tape. Instead of changing incentives, perhaps one could remove incentives altogether. Or insist on replicability statements in grant applications and research papers. Or emphasise collaboration, not competition. Or insist on preregistration of protocols. Or reward better pre and post publication peer review. Or improve research training and mentorship. Or implement the recommendations from our Series on increasing research value, published last year. One of the most convincing proposals came from outside the biomedical community. Tony Weidberg is a Professor of Particle Physics at Oxford. Following several high-profi le errors, the particle physics community now invests great eff ort into intensive checking and rechecking of data prior to publication. By fi ltering results through independent working groups, physicists are encouraged to criticise. Good criticism is rewarded. The goal is a reliable result, and the incentives for scientists are aligned around this goal. Weidberg worried we set the bar for results in biomedicine far too low. In particle physics, significance is set at 5 sigma—a p value of 3 × 10–7 or 1 in 3·5 million (if the result is not true, this is the probability that the data would have been as extreme as they are). The conclusion of the symposium was that something must be done. Indeed, all seemed to agree that it was within our power to do that something. But as to precisely what to do or how to do it, there were no firm answers. Those who have the power to act seem to think somebody else should act first. And every positive action (eg, funding well-powered replications) has a counterargument (science will become less creative). The good news is that science is beginning to take some of its worst failings very seriously. The bad news is that nobody is ready to take the first step to clean up the system. Richard Horton richard.h orton@lancet.com

An exciting update from Philip Z.

  Earlier in June, as a result of the recently enacted First Step Act, I sent a motion to my sentencing judge pro se to request him to vacate or re-sentence me based on two reasons:

    First, there was an “affirmative defense” included in the law under which I was prosecuted which allowed for a parent to remove a child when there was an incidence of abuse. This means I should never have been convicted based on the merits of the law.

    Secondly, affidavits verifying evidence of such abuse had come into my hands within several months of coming to prison, which I have subsequently forwarded to Judge Arcara.

    This past week, Judge Arcara requested a response to my motion to vacate or re-sentence based on my motion to the U.S. government. The prosecution has until August 3 to respond.

    Keep in mind that the alleged abuse of Isabella Miller was never brought out in my nearly two-week trial. The jury only heard it alluded to in passing perhaps one time, but it would have been easily missed. Many of the hundreds of people who wrote Judge Arcara prior to my sentencing alluded to it, however, as did I at the day of my sentencing. This judge, from what I was told, has never allowed anyone convicted in his courtroom, in approximately 30 years, to go free on bail pending appeal. The probation officer was asking for an eight-year prison sentence and a $50,000 fine.

    But on the day of the sentencing, it was very evident God had moved on his heart. My sentence was three years (the minimum), and the fine a mere $200. He several times mentioned the hundreds of letters and the fact that he had read many, if not most of them, twice. Then, on the next day, he allowed Timo Miller, my co-defendant who had already served eight months in prison, to go free at his sentencing, rather than making him serve more time. He then explained to Buffalo News in an interview something to the effect that we were good people who got caught up in something we shouldn’t have.

    In light of my motion, and these two issues I’ve brought to his attention, please pray with our family diligently that God will indeed speak to his heart once again, and send me home in August. I will be in my eighth month of incarceration.

    Pray that he will recognize the injustice of my case, which would have never happened had it not been for the political nature of it. I am asking God to speak to his heart and that he will heed God’s nudge to say, enough is enough.

    I do not regret this opportunity to spend these months in prison. I believe God has used it in my life and others. Has it been easy? No. Was it as bad as I expected? No. Am I glad for the experience? Yes. Did I learn a lot from the experience? Yes. Would I have chosen to go through it in hindsight? Ask me in a year or two or three. Has God used it to His glory? I believe so, particularly if it encourages others to stand firm for Christ and not cower in the face of persecution.

    Please pray with me and believe in faith, that God will once again touch the heart of Judge Arcara, and not just vacate my sentence, but also dismiss the criminal charges against Lisa Miller, who certainly does not deserve to be a fugitive.

    Thank you so much for your many prayers and support up until this point. God has been moving. Two weeks ago, a judge in the western district of Virginia threw out the lawsuit against us filed by Janet Jenkins. Judge Arcara’s favorable ruling at this time would definitely assist in aiding us in winning the civil lawsuit filed against us by Janet Jenkins in Vermont.

    I know that my God is mightier than all the lawyers of the Southern Poverty Law Center combined. In Him alone there is victory!

    Please pray with us diligently over the course of the next month for Judge Arcara to allow me to go home in August.

In His Service,
Philip

To my nineteen followers, and the thousands of others who read this blog

Well, the handwriting is on the wall. Google had already taken over this platform when I started this blog in 2004, but it took a few years for the new paradigm to take effect. Then, overnight, I dropped from 25 hits a day to 12, and from second or third hit on a search to second or third page.
I always use Bing for searches now, and I've recently noticed that a lot of my lost hits from Google are being replaced by Bing; for one recent search, my website was the fifth one down the page.
So for now, I'll limp along, and maybe even see a brief resurgence; but sooner or later the powers that be will see fit to shut me down. By then I hope to have transferred these hundreds of pages of content back down to my computer, so they won't be lost forever. I may even end up making a book out of some of them; fat chance, though, of Amazon letting me use them to sell it.

Or maybe, just maybe, this may turn out to be another one of my famous failed prophecies. We hope.