One of my continuing goals in China is to find traditional Chinese medicines and practices that I can integrate into my Western, allopathic-style family practice. As I’ve reported a few times, I’ve been struggling to find Chinese herbal medicines that I am comfortable prescribing, usually due to lack of evidence for a clear benefit.
As for acupuncture, there are actually a lot more well-designed studies done on this field when compared to Chinese herbs, mostly because Europe and the West have become very interested over the last couple decades and are funding better clinical trials. The evidence trail is building, which is great for everyone. So, what works, and what doesn’t?
The New York Times has a very readable article last week which covers this issue. The gist of the article discusses the most recent, well-designed studies on back pains and other disorders; the studies usually show no difference between acupuncture and placebo (usually a “sham” needle that doesn’t penetrate the skin). In other words, the best studies lately are usually showing that most of the perceived effect from acupuncture is simply a placebo effect.
These recent findings are by no means surprising, as most of the best studies have been reporting similar conclusions for many years. The best collection of acupuncture research is from the Cochrane Review Group, which performs the world’s most rigorous reviews of all treatments, including alternative medicines. They have an outstanding collection of the best literature on acupuncture which details the best evidence regarding a number of acupuncture treatments. Unfortunately, as many other researchers have noted, the only treatments for which acupuncture is even mildly effective are for musculoskeletal problems and nausea. There is some evidence of effectiveness for treatments for IVF, as well as insomnia. But many well-designed studies have specifically shown no benefit over placebo for a host of other treatments. The Cochrane group also commonly finds that the quality of studies is very poor and cannot recommend either for or against.
Readers should be aware that this is not simply a Western bias against alternative medicines; in April a group from Beijing Hospital published in a Chinese journal a similar literature review of acupuncture for insomnia, and couldn’t even make a conclusion because the studies were so poor:
Regarding the assessment of the therapeutic effect, measuring scales are often adopted in overseas studies, while in domestic researches, self-drawn standards are frequently used. In conclusion, there have had no high-quality clinical trails about acupuncture treatment of primary insomnia in China at the present, and the related evaluating methods could not definitely confirm the efficacy of acupuncture in relieving insomnia. Therefore, a strict and scientific clinical trail scheme being in line with evidence-based medicine is urgently needed in the coming studies on acupuncture treatment of primary insomnia.
The Cochrane’s reports cover a lot of therapies that I’m sure many readers have tried. Here’s an example of Cochrane’s findings on the most common acupuncture treatment — low back pain:
Thirty-five RCTs covering 2861 patients were included in this systematic review. There is insufficient evidence to make any recommendations about acupuncture or dry-needling for acute low-back pain. For chronic low-back pain, results show that acupuncture is more effective for pain relief than no treatment or sham treatment, in measurements taken up to three months. The results also show that for chronic low-back pain, acupuncture is more effective for improving function than no treatment, in the short-term. Acupuncture is not more effective than other conventional and “alternative” treatments. When acupuncture is added to other conventional therapies, it relieves pain and improves function better than the conventional therapies alone. However, effects are only small. Dry-needling appears to be a useful adjunct to other therapies for chronic low-back pain.
For depression:
“…Thirty trials, and 2812 participants were included in the review and meta-analysis, however there was insufficient evidence that acupuncture can assist with the management of depression.”
For migraines:
In the four trials in which acupuncture was compared to a proven prophylactic drug treatment, patients receiving acupuncture tended to report more improvement and fewer side effects. Collectively, the studies suggest that migraine patients benefit from acupuncture, although the correct placement of needles seems to be less relevant than is usually thought by acupuncturists.
For insomnia:
Seven studies were eligible for inclusion in the review, involving 590 participants. The studies were of low methodological quality and were diverse in the types of participant, acupuncture treatments and sleep outcome measures used, which limited the ability to pool the findings and draw conclusions. Currently there is a lack of high quality clinical evidence supporting the treatment of people with insomnia using acupuncture. More rigorous studies are needed to assess the efficacy and safety of various forms of acupuncture for treating people with insomnia.
Other reputable sources of evidence include the National Center for Complementary and Alternative Medicine. Also, the excellent evidence-based blog The C.A.M. Report has a section on acupuncture studies. Another blog, Science-Based Medicine, has a section on acupuncture. And I’ve started to read a couple of recently published books which also take a rigorous, evidence-based review of alternative medicines, and I recommend them to my readers. One is called Trick or Treatment: The Undeniable Facts About Alternative Medicine; the other is called Snake Oil Science: The Truth About Complementary and Alternative Medicine. Both can be downloaded via Amazon.com as an e-book. The New England Journal of Medicine has a book review of Trick or Treatment.
Does It Even Matter What The Science Shows?
Many readers may remark that complementary medicine, even if it is just a placebo effect, is still better than nothing. And I think that’s partly true; the placebo effect is simply a person believing and hoping they will get better. It’s an amazing and profoundly humbling revelation that simply believing in healing can set off a biochemical cascade which can boost your immune system and help your body to heal itself. The placebo effect is proven time and again in every study ever done. It’s called faith, and it’s crucial in any illness and is crucial just for living. Here’s a nice quote from the New York Times article:
…acupuncture believers say it doesn’t really matter whether Western scientific studies find that the treatment has a strong placebo effect. After all, the goal of what they call integrative medicine, which combines conventional and alternative treatments like acupuncture, is to harness the body’s power to heal itself. It doesn’t matter whether that power is stimulated by a placebo effect or by skillful placement of needles.
“In general in integrative medicine, when patients are involved in their healing process, they have a tendency to do better,” said Angela Johnson, a practitioner of Chinese medicine at Rush Children’s Hospital in Chicago who is conducting a pilot study of acupuncture to relieve pain in children. “I believe that’s part of the reason why they get better.”…
But one of the problems of relying on alternative medicines (which probably are a placebo effect) would be that you are losing valuable time in getting proper, effective treatment for something which may be serious. Also, you will potentially be spending a lot of your personal money for treatments no better than a placebo.
My Bottom Line
I personally feel that if a patient wants to try acupuncture for musculoskeletal problems like back pains, then they are welcome to try. It’s usually very safe, and sometimes can help; it’s also an interesting experience to do at least once. For almost all other treatments, I would try to pull up Cochrane Database evidence for such treatment and show my patients, and let them make their own decisions.
Hopefully by now, my long-term readers are convinced that evidence-based medicine is the most proper method of testing therapies, and that all treatments of any culture’s medical systems should be tested and proven with rigorous clinical trials. So, I hope we are all open-minded enough to believe that when the best trials show no benefit for a specific treatment — whether mainstream or alternative — then doctors shouldn’t recommend that, and consumers should think twice before wasting their money on it. On the flip side, doctors such as myself should be open-minded and recommend alternative treatments that are proven to work. There just aren’t that many…yet?
UPDATE 2024: Please follow me at my new website, DrSaintCyr.com. Also my new YouTube channel youtube.com/@drsaintcyr
I arrived in China with a Sudeck syndrome on my left leg after a surgery and a 2 months immobilization in stress.
My Swiss doctor had given me some medicine to cure the dysfunction of the sympathetic nervous system, my physio had told me to have water therapy (cold and hot water on my leg 15min per day), it did not change much. After 3 weeks, my surgery was well but I could not start the physio without a lot of pain which would make my leg swell.
10 acupuncture sessions helped me start the physiotherapy without the usual pain and swelling. The first 3 session I had the feeling my leg was waking up, as if I had been standing on it and cutting the blood stream for a too long time.
Maybe what you learned in med school and residency was a little bit biased. I suggest you might take a look at your own paradigm first.
Why So Many (Medical) Studies Based On Statistics Are Wrong
Published by Briggs at 5:48 am under Climatology,Statistics
This was inspired by the (unfortunately titled) article Lies, Damned Lies, and Medical Science, publishing in this month’s Atlantic (thanks A&LD!).
The article profiles the work of John Ioannidis, who has spent a career trying to show the world that the majority of peer-reviewed medical research is wrong, misleading, or of little use. Ioannidis “charges that as much as 90 percent of the published medical information that doctors rely on is flawed…he worries that the field of medical research is so pervasively flawed, and so riddled with conflicts of interest, that it might be chronically resistant to change—or even to publicly admitting that there’s a problem.”
“The studies were biased,” he says. “Sometimes they were overtly biased. Sometimes it was difficult to see the bias, but it was there.” Researchers headed into their studies wanting certain results—and, lo and behold, they were getting them. We think of the scientific process as being objective, rigorous, and even ruthless in separating out what is true from what we merely wish to be true, but in fact it’s easy to manipulate results, even unintentionally or unconsciously. “At every step in the process, there is room to distort results, a way to make a stronger claim or to select what is going to be concluded,” says Ioannidis. “There is an intellectual conflict of interest that pressures researchers to find whatever it is that is most likely to get them funded.”
Most medical studies—and most studies in other fields—rely on statistical models as primary evidence. The problem is that the way these statistical models are used is deeply flawed. That is, the problem is not really with the models themselves. The models are imperfect, but the errors in their construction are minimal. And since (academic) statisticians care primarily about how models are constructed (i.e. the mathematics), the system of training in statistics concentrates almost solely on model construction; thus, the flaw in the use of models is rarely apparent.
Without peering into the mathematical guts, here is how statistical studies actually work:
Data are gathered in the hopes of proving a cherished hypothesis.
A statistical model is selected from a toolbox which contains an enormous number of models, yet it is usually the hammer, or “regression”, that is invariably pulled out.
The model is then fit to the data. That is, the model has various drawstrings and cinches that can be used to tighten itself around the data, in much the same way a bathing suit is made to form-fit around a Victoria’s Secret model.
And to continue the swimsuit modeling analogy, the closer this data can be made to fit, the more beautiful the results are said to be. That is, the closer the data can be made to fit to the statistical model, the more confident that a researcher is that his cherished hypothesis is right.
If the fit of the data (swimsuit) on the model is eye popping enough, the results are published in a journal, which is mailed to subscribers in a brown paper wrapper. In certain cases, press releases are disseminated showing the model’s beauty to the world.
Despite the facetiousness, this is it: statistics really does work this way, from start to finish. What matters most, is the fit of the data to the model. That fit really is taken as evidence that the hypothesis is true.
But this is silly. At some point in their careers, all statisticians learn the mathematical “secret” that any set of data can be made to fit some model perfectly. Our toolbox contains more than enough candidate models, and one can always be found that fits to the desired, publishable tightness.
And still this wouldn’t be wrong, except that after the fit is made, the statistician and researcher stop. They should not!
Consider physics, a field which has far fewer problems than medicine. Data and models abound in physics, too. But after the fit is made, the model is used to predict brand new data, data nobody has yet seen; data, therefore, that is not as subject to researcher control or bias. Physics advances because it makes testable, verifiable predictions.
Fields that make use of statistics rarely make predictions with their models. The fit is all. Since any data can fit some model, it is no surprise when any data does fit some model. That is why so many results that use statistical models as primary evidence later turn out to be wrong. The researchers were looking in the wrong direction: to the past, when the should have been looking to the future.
This isn’t noticed because the published results are first filtered through people who practice statistics in just the same way.
Though scientists and science journalists are constantly talking up the value of the peer-review process, researchers admit among themselves that biased, erroneous, and even blatantly fraudulent studies easily slip through it. Nature, the grande dame of science journals, stated in a 2006 editorial, “Scientists understand that peer review per se provides only a minimal assurance of quality, and that the public conception of peer review as a stamp of authentication is far from the truth.” What’s more, the peer-review process often pressures researchers to shy away from striking out in genuinely new directions, and instead to build on the findings of their colleagues (that is, their potential reviewers) in ways that only seem like breakthroughs…
Except, of course, for studies which examine the influence of climate change, or for other studies which are in politically favorable fields: stem cell research, AIDS research, drug trials by pharmaceuticals, “gaps” in various sociological demographics, and on and on. Those are all OK.
Incidentally, predictions can be made from statistical models, just like in physics. It’s just that nobody does it. Partly this is because of expensive (twice as much data has to be collected), but mostly it’s because researchers wouldn’t like it. After all, they’d spend a lot of time showing what they wanted to believe is wrong. And who wants to do that?
What is evidence based medicine anyway? Why is the evidence constantly changing? I think the fundamental problem with all the emphasis on evidenced based practice is remembering who is funding this evidence to come about.
Think of Chinese medicine, especially acupuncture which doesn’t have the drug company’s money behind it to help find this evidence base practice. One thing that is always forgotten when talking about Chinese medicine in terms of evidence based practice is the lack of funding behind the research. Coming from a developing country, the Chinese government has higher priorities for their money then researching and then educating the west on their traditional medicine. Most Chinese already trust it and don’t need this evidence based practice to use it. I wonder without this funding, how is the research miraculously going to come about?
Have you ever wondered, if as much money and support by large pharmaceutical was poured into Chinese medicine per year as it is into western medicine…. we might start seeing some different results? I have!
It’s simply not true to suggest that there is a “lack of funding” behind research into TCM. There are major Chinese hospitals and researchers, and dozens of institutes and magazines within China dedicated to researching TCM. And it certainly isn’t true that “most Chinese already trust it and don’t need this evidence based practice”. In my experience, the average Chinese person, when very sick, would still prefer Western medicines and hospitals. And more than a few Chinese, especially our blog readers on this website and all over Weibo, are quite skeptical of much of TCM. And on my Weibo, by far the #1 concern among my Chinese readers is to find evidence-based recommendations for good health care — whether Western or TCM.