Regarding your second point, the fact that most research does not show an effect means very little when the research that does has such incredibly strong results. Science is not a democracy where the experiments get to vote equally. Some votes count stronger than others. I refer you to the methodology of science if this confuses you. But you can see the same effect (as I mentioned before) in the cern experiments where thousands if not millions of particles are shot at each other. Most of these experiments are failures that’s why they built an enormous computer system to wade through the research and find experiments that did show something. Claiming that the few collisions don’t mean anything because there are many more misses is not correct.

Regarding your third point, in the experiments I referenced to almost all homeopathically treated individuals experienced sharp improvements compared to the control group. That would indicate that at least in some situations it’s a highly effective medicine.

I have stated these three points before. I have added nothing new to this comment.

Let me make it clear again that I’m not saying homeopathy works. I have no personal investments in it, I never used it and don’t have personal experiences with it. I just get a bit ticked off when under the guise of science things are being stated that are just not true. I happen to be educated enough to be able to judge and evaluate these statements but others aren’t.

I have the greatest respect for mr Plait and in my opinion he’s correct in demanding criticism, certainly regarding homeopathy. But that should at least mean that we should be critical of what we say ourselves. He said there was no evidence… He was wrong… I showed it. That’s the lot of it. No need to buy stocks in homeopathic medicine. But letting them do their research without ridicule whilst demanding proper application of scientific method sounds about fair to me.

Just think about it for a moment. If there is actual evidence in public research databases right now, and I gave the links to them, while the skeptics claim there isn’t. What does this say about the skeptics?

I think that the key concept here is repeatability. You can quote one article with a very high p-value, but we do not know whether that demonstrates an actual effect, or an undiagnosed error on the part of the researchers. The plausible explanation, in the absence of any sound theoretical basis for why we should expect homeopathy to work at all, given that the entire body of scientific theory predicts that it should not, and in the absence of a well-grounded body of evidence that supports the homeopathic hypothesis, is that someone buggered up the study.

To hijack your example, scientific tests of homeopathy may be mistaking a dust mote on the lens for Daimos.

We can test this by running the study multiple times in multiple labs with multiple safeguards against error. Once we have an appropriate body of positive studies that collectively eliminate the chance that the effect is due to methodology, we can entertain the possibility of accepting an hypothesis that would contradict what we know about physics, biology, and chemistry (and contradicts itself, what with the ocean of unregulated homeopathic preparations a person must be drinking each time they lift a glass of water, but which somehow have no effect). However, there are reasons not to bother. First is the general implausibility of the hypothesis. We would be better off performing research that is based on theories which are known to obtain results.

Second, the research to date is mostly negative. The chances that these few positive results are evidence of an enormous effect that we have somehow merely overlooked are very slim. Rather, it is more likely that we will discover that the odd ones out are the ones in error, not the majority. Reviews have found that there is a correlation between the quality of the methodology used in a homeopathic study and the chance of a negative result. We would expect to see the opposite if there were really an effect.

Third, even if there is some kind of effect, we might note that since it is apparently so hard to find even under controlled conditions, the chance that it is biologically relevant is probably null. If clinical laboratories cannot reliably squeeze the blood of significant results from the stone of homeopathic treatment, what hope does the homeopathic practitioner have of effecting an improvement–much less a cure–in a patient?

You’re mostly repeating the request for evidence. Please turn your attention to my comment #67 I provided a link to an article there answering your exact question. With the added bonus that they are double-blind, highly significant, and taking the placebo effect into account! As I said in that message there are 2 links on that page to two other experiments just like it. The one I linked to has a significance of p<0.0001. Even you must grant that that is significant.

As I told Geek, this does not mean the experiments are valid. And we must certainly not just accept homeopathy for a fact just on the basis of this. The point is that they serve as evidence. So the statement that there is no evidence is false… And that is the point I am making. Not that homeopathy works. Because I’m not convinced myself. I just see a valid case for homeopathy in statistical terms. I want to see more credible ideas on mechanism. But statistically speaking they have a case.

You say that an effect measured with p<0.05 that is only reproducible a 10th of the time is not significant. First of all, it’s not the effect that has significance. It’s the result of the experiment that has significance. And I’m sorry but p<0.05 is considered highly significant. Mind you the experiment I linked to has a significance of p<0.0001 which is a few orders of magnitude better…

The fact that it is not reproducible all the time is not an issue. I remind you of particle physics where literally up to thousands or even millions of collisions need to be caused and recorded in the hope of finding some rare interaction or particle. The statistics clearly allow for this. I also gave a binocular example that basically already answered this point.

It is important to note that in the example I gave I said you have to do a sufficient amount of valid experiments. So it’s not just the first batch of 10 with one hit after which you quit.

This is not an advanced topic. Significance validity and reliability are core concepts of statistics. I don’t know your background and I don’t mean to offend I just want to avoid getting hung up on words. If you’re unsure on these concepts look them up, without knowing their meaning the whole discussion is going to be mighty confusing. In daily life we tend to mix up reliability and validity quite a bit. And you seemed to do that once before. Significance in life also doesn’t relate well to the mathematical meaning of significance in statistics.

Like I said before its true that there is no convincing mechanism we can use to explain this. However, we cannot deny that there is statistical evidence that such a mechanism could exist.

A question though, you clearly know how to quote.. Could you spend a few sentences on teaching this noob the trick?

I’m sorry I find it hard to discover the points you’re trying to make in your comment. And you mention a few things that are just wrong.

Electroconvulsive shock therapies are brutal this is true, though not as bad as one would naively expect. You basically only use it under the conditions you stated. In other words as a life saving intervention. You don’t just put a man under a heart defibrillator either, unless the situation is acute. In which case they are also the best methods we have.

Actually routine cardioversion is commonly used as an outpatient procedure. It is the same defibrillator, the difference is that the shock is timed to land on the QRS complex, rather than be delivered randomly.

So apart from my failure to understand the relevance, you’re basically stating the obvious here.

You also state: “There is a ridiculously remote chance that there is a good reason for homeopathy to completely fail so many studies, but somehow still work.” This is not true. A failed experiment fails to pick up an effect, it’s like looking at the sky through binoculars. You’ll fail to see mars if you look randomly. You’ll usually fail to see mars if you don’t know exactly where to look. And you should be able to see mars regularly if you know where to find it. Now replace mars with Deimos and keep the binoculars. I think it’d be virtually impossible to look at the right spot in the sky, and see Deimos. If you repeat that experiment a million times.. Predictably and reliably fail to see Deimos. And then conclude Deimos does not exist, you’d be wrong.

When it comes to medical treatment, if we cannot find a difference between our study treatment and placebo on a consistent basis, we should conclude that whatever effect might be there is so insignificant that it is not any better than an inert substance, which pretty much defines distilled water as a treatment.

The opposite, that if you see Deimos the chance of it not actually being there, is much much smaller.

I am aware of that. If only they could show that the homeopathic Deimos is there. So far it seems to be less significant than Oakland – no there there. And there really isn’t any reason to expect that to change.

The experiments are designed to guard against false positives. We only add positive statements to our understanding of the world. And reject outcomes if they fail our demands for reliability. We don’t add negative statements. Because: while it only requires a valid demonstration to show a positive, it is virtually impossible to show a negative.

The inability to find an improved outcome is something that would encourage a scientist, interested in finding effective treatments, to move on to something that can produce positive results. Homeopathy does not encourage any optimism.

In medicine, which homeopathy claims to be, the burden of proof is on those proposing that they have an effective treatment.

Homeopathy needs to show a benefit to patients or go away.

It is long past time for homeopathy to go away.

So if an experiment fails to show an effect. It does not serve as evidence that the effect is absent.

Now you are the one stating the obvious. Maybe you should read the book by Dr. Penston. While he is critical of the waste and misuse of science by the drug companies, his criticism is just as valid for homeopathy. We need to have some effect to justify continuing to subject patients to these treatments.

It is the responsibility of the drug pusher to show that the treatment is effective.

In the absence of evidence of benefit, it is wrong to continue to subject patients to these experimental treatments.

Now this might sound like picking side with the homeopathy guys. But I’m not, this is how we do science. And you’d learn this stuff on methodology 101 of most educations that use statistics. This is not negotiable as far as I am concerned.
It’s like I said before, if you measure with a significance of for example p<0.05. You don’t need every other experiment to succeed. If a one in 20 chance occurs once every 10 trials. And you do sufficient and valid experiments. Then there is a clear and demonstrated effect that no statistician would argue with. So here is a clear situation where even failing to show an effect in most studies we’d still be required to accept the reality of the measured effect in the long run.

An effect that is so insignificant, that it is only occasionally there, is not worth it for the patient. Such unpredictable results suggest that something else is going on. Until there is a consistent effect, we need to protect patients. The burden of proof in medicine is on the one proposing the treatment. Homeopathy does not meet that standard.

“I can find out what is likely to be effective with conventional medicine. I can find out medicine’s success rate. I can find out the conditions it has been used to treat. I can find out the side effects. I can have a very good idea of what to expect. For a patient, a doctor can point all of that out to the patient.”

Replace medicine with homeopathic treatment, and doctor with homeopath and the sentence is as true as before, in spite of your claims of the opposite.. They don’t know how it works. But the success rate is measurable. The conditions it’s supposed to treat are known. The side effects if any are known. People do know what to expect and the homeopath can explain this to the client.

Please provide some reproduceable studies that show a positive effect in treating patients.

We’re not talking about all that though, we’re talking about efficacy and the state of related experiments, not protocol in the doctors office.

If it is going to be used on patients we need to be able to show that is likely that the treatment is causing patients to get better. In the absence of that, it is just another unproven experimental treatment.

I said “According to scientific thought unless a number of experts cannot point out the errors in an article or experiment we must assume that it’s outcome is correct.” You said “No. We have studies that can only be reproduced now and then. ”

Please note that you’re not contradicting me even if you think you are. I speak of validity and you speak of reliability. These are two very different subjects. Validity can be interpreted as an experiments ability to show an effect if it exists. Reliability means the experiment can be relied upon to do it again if it is repeated.

Do we really want to treat patients with treatments based on valid, but unreliable research?

Again, in medicine, the burden is on the person selling the treatment to show that it works.

Homeopathy fails.

An analogy would be a volt meter. It has an error margin which is it’s reliability. And doing a valid measurement of the voltage in your wall socket implies measuring the wall socket. An invalid reading would be measuring your neighbors wall socket. Even if the results seem to agree and are probably correct the experiment is still not valid.

A reliable experiment that shows effect but that is not valid could be similarly measuring the wrong signals. A valid experiment could fail to pick up an effect that is too weak. Reliability of an experiment can be measured by repeating it. Validity can only be ascertained by careful analysis by experts. If experts can find no fault this does not necessarily mean the experiment is valid. But as far as we can tell at that point it IS valid. We cannot determine validity on gut feeling or by repeating an experiment. We need peer previews for that. People trained in methodology who are able to point out potential flaws or trouble.

If we were just playing around with wall current, we could keep going on indefinitely with only opportunity loss. This is supposed to be a medical treatment. When we cannot produce a repeatable benefit, we need to stop subjecting patients to this mistreatment.

When there is repeatable evidence of a benefit to patients, then it will be worth paying some attention to homeopathy, which is just another failed experimental treatment.

Until then homeopathy is only useful as a punchline, and a very good punchline.

Electroconvulsive shock therapies are brutal this is true, though not as bad as one would naively expect. You basically only use it under the conditions you stated. In other words as a life saving intervention. You don’t just put a man under a heart defibrillator either, unless the situation is acute. In which case they are also the best methods we have. So apart from my failure to understand the relevance, you’re basically stating the obvious here.

You also state: “There is a ridiculously remote chance that there is a good reason for homeopathy to completely fail so many studies, but somehow still work.” This is not true. A failed experiment fails to pick up an effect, it’s like looking at the sky through binoculars. You’ll fail to see mars if you look randomly. You’ll usually fail to see mars if you don’t know exactly where to look. And you should be able to see mars regularly if you know where to find it. Now replace mars with Deimos and keep the binoculars. I think it’d be virtually impossible to look at the right spot in the sky, and see Deimos. If you repeat that experiment a million times.. Predictably and reliably fail to see Deimos. And then conclude Deimos does not exist, you’d be wrong.

The opposite, that if you see Deimos the chance of it not actually being there, is much much smaller.

The experiments are designed to guard against false positives. We only add positive statements to our understanding of the world. And reject outcomes if they fail our demands for reliability. We don’t add negative statements. Because: while it only requires a valid demonstration to show a positive, it is virtually impossible to show a negative.

So if an experiment fails to show an effect. It does not serve as evidence that the effect is absent. Now this might sound like picking side with the homeopathy guys. But I’m not, this is how we do science. And you’d learn this stuff on methodology 101 of most educations that use statistics. This is not negotiable as far as I am concerned.

It’s like I said before, if you measure with a significance of for example p<0.05. You don’t need every other experiment to succeed. If a one in 20 chance occurs once every 10 trials. And you do sufficient and valid experiments. Then there is a clear and demonstrated effect that no statistician would argue with. So here is a clear situation where even failing to show an effect in most studies we’d still be required to accept the reality of the measured effect in the long run.

“I can find out what is likely to be effective with conventional medicine. I can find out medicine’s success rate. I can find out the conditions it has been used to treat. I can find out the side effects. I can have a very good idea of what to expect. For a patient, a doctor can point all of that out to the patient.”
Replace medicine with homeopathic treatment, and doctor with homeopath and the sentence is as true as before, in spite of your claims of the opposite.. They don’t know how it works. But the success rate is measurable. The conditions it’s supposed to treat are known. The side effects if any are known. People do know what to expect and the homeopath can explain this to the client.

We’re not talking about all that though, we’re talking about efficacy and the state of related experiments, not protocol in the doctors office.

I said “According to scientific thought unless a number of experts cannot point out the errors in an article or experiment we must assume that it’s outcome is correct.” You said “No. We have studies that can only be reproduced now and then. ”

Please note that you’re not contradicting me even if you think you are. I speak of validity and you speak of reliability. These are two very different subjects. Validity can be interpreted as an experiments ability to show an effect if it exists. Reliability means the experiment can be relied upon to do it again if it is repeated.

An analogy would be a volt meter. It has an error margin which is it’s reliability. And doing a valid measurement of the voltage in your wall socket implies measuring the wall socket. An invalid reading would be measuring your neighbors wall socket. Even if the results seem to agree and are probably correct the experiment is still not valid.

A reliable experiment that shows effect but that is not valid could be similarly measuring the wrong signals. A valid experiment could fail to pick up an effect that is too weak. Reliability of an experiment can be measured by repeating it. Validity can only be ascertained by careful analysis by experts. If experts can find no fault this does not necessarily mean the experiment is valid. But as far as we can tell at that point it IS valid. We cannot determine validity on gut feeling or by repeating an experiment. We need peer previews for that. People trained in methodology who are able to point out potential flaws or trouble.

You mention anti-depressants. I am not familiar with the research. I read Toxic Psychiatry and it made many good points. When it comes to ECT (ElectroConvulsive Therapy), I will never accept that frying my brain, while I am still using it, is beneficial. I see this as something that is effective only if you ignore a lot of side effects and suicide is the alternative.

But when you extrapolate from medicine not understanding psychology to homeopathy, you are dealing with apples and oranges. You have brain function (neurology), that we do not understand, even now. Then you have mind function (psychology), which may be just a function of neurology, but we do not know. We prefer to think that we control our actions, but there is research that suggests otherwise. That may sound bad for psychology, but you will always have plenty of business. No need to have some Vogons blow up Earth.

Yes. Doctors are notorious for refusing to accept abundant evidence that something works, or evidence that something doesn’t work. So, it is easy to pull those examples out. The scientific method has demonstrated effectiveness in the mean time.

Today, medicine is much more science based. Since 1941, there is the ability to completely cure a disease. Many of the people in medicine before then have had trouble understanding the kind of change that was. Penicillin was so good, it made all of the rest of the medicine cabinet look bad. This was actually a good thing.

Today, you will find that many more doctors are familiar with the scientific method and demand evidence of effectiveness before using treatments on their patients.

Homeopathy? Not so much. Here is the story of the How Dr. Hahnemann Stole Medicine!

Then he got an idea. An awful idea. Dr. Hahnemann got a wonderful, awful idea! “I know just what to do!”

At the time (late 1700s) medicine was not curing people. Any old wacky idea was as good as any other. As long as you did not give patients too much of a poison, they might get better in spite of you. Conventional medicine has changed tremendously since then. Homeopathy stays the same. That wonderful, awful idea? Like cures like. Dr. Hahnemann found that this worked in a few cases, but he extrapolated from those cases to everything.

So, consider the theory behind this, that which does not kill the patient makes the patient stronger, but by diluting it, there is much less chance of the patient dropping dead – at least much less chance of them dropping dead from the homeopathy. That is the idea. Add some succussions and dilute until pure water or some concentration of alcohol.

Modern homeopathy has no more understanding of why they think it works than that 200 year old idea.

We did not know how aspirin worked for the longest time.

But we did know that there was an effect. Knowing there is an effect and understanding the basis for the effect are entirely different. You can see that aspirin has many effects. These effects are measurable. These effects are repeatable. Consistently.

According to scientific thought unless a number of experts cannot point out the errors in an article or experiment we must assume that it’s outcome is correct.

No.

We have studies that can only be reproduced now and then.

We do not need to know what the problem is, only that there is something very odd about such widely divergent results. It suggests that we are not testing what we think we are testing.

I assume you have done some teaching and are familiar with testing students. If a student is taking multiple choice tests, and these studies are analogous to multiple choice tests, what do you conclude about the knowledge of the student when the results are all over the place?

Maybe he/she has MPD (Multiple Personality Disorder). Not likely, since the occurrence of MPD is probably much higher in fiction or in theories, than it is in real people.

Maybe something was affecting the testing conditions. He/she may be very distractible. Special testing situations may be needed to allow the student to demonstrate knowledge of the material. However, this may be a subject that cannot be practiced in a fortress of solitude. If the testing conditions are not realistic for the use of what is being tested, you have someone who is only good at taking tests with such tight controls that there may be no real world application of that knowledge.

In the real world, when you administer a test, if the person taking the test produces widely different results, that are sometimes great and sometimes horrible, do you draw any conclusions about the knowledge of the student?

What about some sort of target shooting? I will use darts for this example. What if the person occasionally throws a bunch of darts in higher scoring parts of the dart board, but also has bunches that don’t even hit the dart board?

Would you use the term ambiguous?

If these were shots with a crossbow, and some look good, while others seem to exhibit more amusement value, than talent. Would you allow this person to try to shoot an arrow off of your head? While the analogy to medicine is strained on this one, I am trying to get you to just look at this from several different angles.

Then there is an excellent book Fiction and Fantasy in Medical Research. The Large-Scale Randomised Trial by James Penston (2003, The London Press, London England. ISBN: 0-9544636-1-7). If you haven’t read it, I recommend it. Only 144 pages, so not even a long book. He spends the first part of the book on cause and effect and how philosophers have approached cause and effect, so that part should be a review.

The big part is his conclusion. If we need huge studies to show any effect, are we measuring what we think we are measuring and is it an effect that is worth paying any attention to. That is oversimplified, but I am trying to reduce it to a haiku.

There is a ridiculously remote chance that there is a good reason for homeopathy to completely fail so many studies, but somehow still work.

There is a ridiculously remote chance that the theory of homeopathy coincided with some completely unknown method, that is still completely unknown, but somehow still works.

Then there is the part that really is confusing. If it does work, what does it work on? How do we know?

I can find out what is likely to be effective with conventional medicine. I can find out medicine’s success rate. I can find out the conditions it has been used to treat. I can find out the side effects. I can have a very good idea of what to expect. For a patient, a doctor can point all of that out to the patient.

Homeopathy? Not so much.

It couldn’t hurt is often used as a reason to try it, but if it does have an effect and we don’t know what the effect is, how do we know that it couldn’t hurt?