CHOOSE MEMBERSHIP LEVEL

Subscribe to E-Mail Updates

Latest tweets
BECOME A FAN

Sunday
Jul312016

Ben Goldacre on "Bad Trials" - Lecture Recap

Report by Russ Dobler on July 18, 2016 event.

Note:  An earlier version of this article misidentified Ben Goldacre's COMPare group as CONSORT, which is an unrelated set of recommendations for reporting randomized trials. The original article also incorrectly stated that no randomized trials had been conducted on the drug digoxin until recently.

It’s humbling to talk in America,” British academic and bestselling author Ben Goldacre told the New York City Skeptics, “because nobody knows who the fuck I am in America.” 

A packed house at Baruch College on this wet, hot, American summer evening argued otherwise. The physician and writer of the books Bad Science and Bad Pharma was on the continent and in town on July 18 to talk about a specific way in which medical science goes wrong, with bad drug trials.

Tens of thousands of randomized controlled trials (RCTs) are conducted every year, Goldacre said, to avoid the pitfalls of just using observational data to judge a treatment’s efficacy. The heart failure and atrial fibrillation drug digoxin, for example, was prescribed for years to lengthen the lives of cardiovascular patients, based largely on anecdotal reports that it worked. Later metanalyses of the existing research instead showed that digoxin, in reality, had no positive effect on mortality, although it may decrease the number of required hospitalizations.

But even the best intentions can yield bad results when trials are conducted improperly. Goldacre came armed with a laundry list of methods with which researchers can manipulate data, knowingly or unknowingly, to fool academia, the press or even themselves. 

About 30% of drugs approved by the U.S. Food and Drug Administration between the years 2000 and 2010 were tested with RCTs, but only against placebos, to see if the treatments worked better than no treatment at all. “That sounds superficially like the right thing to do,” Goldacre said, but the goal should actually be to find out if a treatment works better, and with fewer side effects, than existing drugs.

Of course, there’s room for deception when testing in direct comparison with other drugs, too. Some experimenters might use an unusually high dose of an existing treatment to accentuate its side effects, or a lower dose than needed of the subject drug, to mask its own. On the other side, pumping up the administered amount of a new drug might make it seem more effective than it actually is.

Choosing suitable participants for a trial is important, too, but not always for the reasons researchers think. “All too often, trials are conducted on ideal, perfect patients,” Goldacre said, when real life populations are “messy.” What good is it to bar marijuana-smokers from schizophrenia drug tests when many schizophrenics use it to help cope with symptoms? Some trials of clinical depression drugs don’t even accept people who drink alcohol.

And once a researcher has decided on selection criteria, that person needs to make sure someone else does the selecting. This kind of blinding can prevent experimenters from (consciously or unconsciously) tipping the scales by putting certain people in certain groups. The developer of a drug may want to put an especially sick person into the control group, so as to not make his own drug look bad if the patient doesn’t improve, for instance. Analyses have shown that drugs can seem to have up to 40% greater effects than they actually do if this allocation process is not done properly.

The whole thing becomes even murkier when you’re only testing what Goldacre called “surrogate” outcomes. "In the real world, the outcomes we’re concerned with are pain, suffering and death,” Goldacre said. Why is it then the things most-often tested are only related indicators, like cholesterol level and blood pressure?

“People take a long time to die,” Goldacre said of the practically of directly testing something like mortality. But using surrogate outcomes can leave you still wondering. While diabetes drugs are very good at lowering a patient’s blood sugar, not a single one has been shown to reduce the chances of amputations, heart attack or stroke.

That’s probably still better than switching what outcomes you’re testing for after the trial has begun. “You can give yourself so many opportunities to cherry pick your results,” Goldacre said, which can even lead to thinking that a treatment might be better for one group of people especially. A particularly famous (and poignant) instance was when British medical journal The Lancet insisted the ISIS-2 trial include a subgroup analysis, after the fact, to see if aspirin was more or less likely to prevent heart attacks in certain cohorts. Turns out it’s not very effective for Libras or Geminis.

This kind of "outcome switching" is the particular focus of the Centre for Evidence-Based Medicine's Outcome Monitoring Project (COMPare), of which Goldacre is a part. Between October 2015 and January 2016, the group collectively assessed the findings of every trial published in the world's most prestigious medical journals, to see if their findings were properly reported.

“We had three meetings a week; they were two hours long [each],” Goldacre said. “They were fucking miserable.” The results were perhaps even more painful, as just over half of all pre-specified outcomes were reported, and an average of five new outcomes were added to each study.

Things didn’t get much better when these observations were brought to the attention of those major medical journals. While the BMJ published corrections to the cited studies on their website, The Lancet waited six months to address the issue in print, with responses from the original trialists.

“This is no way to deal with the problem,” Goldacre said.

The Journal of the American Medical Association wanted nothing to do with COMPare at all, and didn’t respond when COMPare’s letters were corrected for being too “vague” and “repetitive.” The New England Journal of Medicine went the extra mile and noted all the mistakes COMPare itself had made. There was no response from the journal when Goldacre and company pointed out, with examples, how the accusation was factually incorrect.

More than half of the 58 total correction letters sent were rejected by journal editors.

In addition to working with COMPare, Goldacre helped found the AllTrials initiative in 2013, an effort to make sure that all clinical trials are reported, as currently only about half are ever published.

Wednesday
Jul132016

Enrico Fonda on "Creative Turbulence: from Leonardo Da Vinci to Quantum Vortices" - Lecture Recap

Report by Russ Dobler on June 25, 2016 event.

When NYU post-doc Enrico Fonda fired up the fog machine on June 25, it wasn’t to commemorate Mötley Crüe’s final performance, or to prepare for the following day’s Pride Parade. It was for science! 

The fluid dynamics physicist shined a laser through the haze he created at the New York City Skeptics lecture at Baruch College to demonstrate the concept of turbulent flow. You might think of turbulence as just that thing that causes white knuckles on bumpy airplane rides, but it also happens to be one of the worst-understood phenomena in all of physics.

“It’s not that there were not enough smart people working on the problem,” Fonda said of the history of turbulence research. Even Nobel laureates Werner Heisenberg and Richard Feynman tried to tackle turbulence and failed. It led mathematician Horace Lamb to famously remark,

"I am an old man now, and when I die and go to heaven there are two matters on which I hope for enlightenment. One is quantum electrodynamics, and the other is the turbulent motion of fluids. And about the former I am rather optimistic.”

Turbulence is a seemingly simple idea – the description of how things like air and water flow when not tightly constrained. It’s apparently governed by some well-known mathematics, like the Euler and Navier-Stokes equations, and it surrounds us every day – not just on airplanes, but even in the way perfume is wafted across a room, Fonda said. Yet the enormous amount of data needed to precisely predict any given turbulent flow makes it more difficult to get a handle on then even more elusive things like the Higgs boson, creating a phenomenon that Fonda says is “more complicated than chaos.”

“Still, it’s not clear exactly how water flows in a pipe, to this day,” Fonda said.

Turbulence is so mysterious that the people who currently work on it can’t even agree what its biggest unanswered question even is. Is it really how to predict the phenomenology from the equations, or should we concentrate on more practical things, like how to reduce drag on vehicles or increase mixing in solutions? Do you even need to understand the equations for that, or would a model suffice? Is it more important to understand how dependent turbulence is on initial conditions, or how it transitions from simpler, laminar flow later on?

And how did Vincent van Gogh get it so right in “Starry Night”? 

The occasionally psychotic painter, who committed suicide in 1890, had an intuitive grasp of turbulent flow, claims physicist Jose Luis Aragón. Aragón says the swirling patterns of light and dark in the famous piece align well with what’s predicted in real-life Kolmogorov scaling, a law that wasn’t developed until the 1940s. The result seems significant, as Edvard Munch’s iconic “The Scream,” with superficially similar swirl patterns, couldn’t pass the same test. Other creators have used geometry in their work, too, like sculptor Anish Kapoor and stained-glass artist Harry Clarke, but Fonda now wants to take things a step further.

“We organized a project that pairs scientist with artists,” Fonda said of his effort to build research results into new artwork from the ground up. The Physics of Art exhibition debuts at Brooklyn’s Pioneer Works facility this fall, and will feature, among others, the work of electronic artist Rafael Lozano-Hemmer, who has a bachelor’s degree in physical chemistry.

Fonda’s helped create some beautiful imagery himself, as part of the team that was able to directly observe the Kelvin waves of quantum turbulence for the first time. Check out a description and video here, and check back with New York City Skeptics for more info on “The Physics of Art” as it’s announced. 

Sunday
Jun262016

Lecture: Ben Goldacre on "Bad Trials"

When: Monday, July 18, 2016 @ 7:00 PM 
Where: Baruch Performing Arts Center, 17 Lexington Ave., Room 306 (The "Skylight Room")

Randomised controlled trials are supposed to be the gold standard, the most "fair tests" of which intervention works best. But sometimes, through cunning or incompetence, trials can be biased by design: so that they overstate the benefits, and hide the side effects. This talk is a greatest hits of bad trial design; in the hands of the enemy, it is a manual for deceit; in the hands of the good, it is vital self-defence.

Ben Goldacre is a doctor, best-selling author, academic and campaigner. His work focuses on uses and misuses of science and statistics by journalists, politicians, drug companies and quacks. His book Bad Science reached #1 in the UK non-fiction charts and has sold over half a million copies worldwide. He has published extensively in all major newspapers and various academic journals, and appears regularly on radio and TV from Newsnight to QI. He has written government papers and reports on evidence based policy, founded a successful global campaign for research transparency, and currently works as an academic in the University of Oxford, where he runs the EBMdataLab building live data tools to make science and medicine better, like OpenPrescribing and OpenTrials. His blog is at www.badscience.net and he is @bengoldacre on twitter.

NYCS tries to keep their lectures free and open to the public, but it does cost money to produce the events. Please consider donating to NYCSkeptics or becoming a member. Suggested donation for this talk is $10.

Event Links