Does regression to the mean explain successful diet programs?

We might remember ‘regression to the mean‘ from those lists of threats to validity (in terms of causal analysis). But when is it actually likely to be a problem for genuine evaluation? In a recent post by Rebecca Goldin on the blog, “Why any ol’ diet will work (if your BMI is high enough): A case study in regression toward the mean” statements were made about its relevance for studies of BMI that I have found very confusing. This is an issue I’ve puzzled over during a particular evaluation and I wonder if the argument in the post stacks up.

My understanding is that regression to the mean is an artefact of the unreliability of measurement. So, for example, a student’s test score might be affected by whether they had a good or bad night’s sleep, a good or bad breakfast, and a good or bad social interaction just before the test, in addition to their actual skills and knowledge in the topic (and in taking tests on the topic). So if we select students who have scored low, implement an intervention, and then test them again, we might find that a number who scored low initially (but whose test scores were lower than their actual level of knowledge) achieve higher scores due to more accurate measurement the second time.

The post discusses the example of blood pressure, which notoriously fluctuates (so a high reading on one day might not indicate much) and coin flipping (which is a random process, so you could get an odd high measure) – and then generalizes to BMI, which doesn’t seem to me to be relevant. Wearing heavier shoes for the first weighing, or having just eaten, might make you a little heavier, but it’s unlikely to be significant if the measurements are being done well (and we have pretty good measures of mass and height). As many of us can testify, BMI doesn’t tend to fluctuate much, and is in fact quite slow to change.

The title of the blog also seems to be contradicted by the conclusion to the post which acknowledges:

What diets do not advertise is that people with the highest BMI people are (on average) losing weight even without a diet – though they may well hover at a very high BMI. For full disclosure: the random variance of weight measurements may in fact be very small.

For people with higher-than-average BMI trying to shed some pounds, regression to the mean is an encouraging thought — but of course, it’s only reduced calorie consumption and exercise that will shed pounds instead of ounces

So is regression to the mean only likely to explain tiny reductions (which would be useful for diets that focused on the number who had lost any weight) but is not likely to be the explanation for large reductions? So does it come down to what we mean by “it works” – any reduction, or a clinically significant reduction? Or is the title misleading?

Any helpful statistical advice on this one?

Comments are closed.