Podcast Episode 240: Obesity Research: Rigor, Reproducibility & Truthful Communication – Dr. David Allison

[music]

Voice-over: Welcome to Sound Bites, hosted by registered dietitian, nutritionist Melissa Joy Dobbins. Let’s delve into the science, the psychology, and the strategies behind good food and nutrition.

[music]

Melissa Joy Dobbins: Welcome to the Sound Bites podcast. Today’s episode is about nutrition and obesity-related research, in particular scientific rigor, reproducibility, and transparency. What does the current body of research say about obesity treatment and prevention? How much weight should those results or outcomes carry given the limitations and weaknesses in research studies?

My guest today is Dr. David B. Allison, the dean of Indiana University Bloomington School of Public Health and a distinguished professor of epidemiology and biostatistics. He has been a researcher funded by NIH continuously for several decades, is an elected member of the National Academy of Medicine, and well-known for his tireless promotion of scientific rigor and the unvarnished, truthful communication of science. He speaks here today on his own behalf, and his views do not necessarily represent those of Indiana University or any other organization. Welcome to the show, Dr. Allison.

Dr. David B. Allison: Thank you. Good to be here.

Melissa: So excited to have you on the podcast. We chatted before. Should I call you Dean Allison, Dr. Allison, Professor Allison? I think, Professor is what we settled on. You wear a lot of hats. You have an amazing background.

David: David is fine, Professor Allison’s fine.

Melissa: Wonderful.

David: I get called worse than that.

Melissa: Well, thank you so much for joining me today. We have some really interesting science and [00:02:00] insights to hear from you on. For the listeners, just want to let you know this episode is not sponsored. We may be submitting it to the Commission on Dietetic Registration for continuing education. Stay tuned if that’s of interest to you.

Now, Professor Allison, I would love to start with hearing more about your background, your education, and the work that you do. Maybe how you got interested in nutrition and obesity research to begin with. Your CV is upwards of 160-plus pages. You have authored more than 660 scientific publications. There’s so much that you have you done and you are currently working on. I would just love to hear what you would like us to know as we transition into this conversation.

David: Well, first of all, thank you for inviting me and being interested in my opinions. I would say that nutrition and obesity-related research are interesting and important scientific topics. They’re a little bit unique, or at least different than some other topics, in that they’re topics that all of us have everyday familiarity with. Almost all of us eat almost every day, and we’ve done that our entire lives. It’s easy to mistake familiarity for expertise.

I think familiarity does give us insights. I often look at my own eating behavior or those around me and draw tentative conclusions. That’s not the same as scientific information. We wouldn’t do that with thinking about an organ system, how the thymus works, or with a molecular basis, how this gene or protein works. Why do we think that we can jump to conclusions about what’s true with nutrition or obesity without the science? As scientists, we need to subject it to the same rigor so that we can really get answers to questions that are important, if we think they’re important, as we do with any other science.

I think that’s something [00:04:00] that’s been challenging for the field as a whole, applying that rigor and also putting aside our personal philosophical views, our personal tastes, our religious views, our preconceptions, our anecdotes, and not mistaking them for scientifically demonstrated things and being on the lookout for that. I think that’s something we all need to work on more. That’s a big part of what I do in my career and what I hope we’ll do as a field, is to say, “Let’s elevate this, let’s treat nutrition questions and obesity-related questions like the same science that any other science is.”

Melissa: Yes. That familiarity, that really resonates. Even as dietitians who are listening, we’re always thinking, “Yes, everybody eats, so everybody thinks they’re an expert in food.” It’s tongue in cheek a little bit. That is a really good point and launching pad for the rest of our conversation. Your research interests go beyond obesity and nutrition, quantitative genetics, not sure what that is, but you’re very interested in clinical trials.

You’re a statistician, you’re interested in research methodology, and obviously, rigor and integrity. I appreciate your perspective as to what makes this obesity and nutrition-related research different. I would love to hear perhaps an overview of the various aspects of research that we need to understand or consider as we’re looking at this complex issue.

David: Sure. Well, I think some of what makes it different, and not completely unique, but a little bit different than some others is, is partly what I’ve already mentioned. It’s the everyday familiarity and it’s the strong connection we have perhaps in part because of the everyday familiarity. That’s an everyday familiarity that’s gone back since the dawn of time, the dawn of humans. It’s familiarity that involves our ancestors, cultural beliefs, [00:06:00] religious beliefs, family, child rearing, economics and so on.

Somebody once said to me, this was my old friend, Fergus Clydesdale, one of the leading food scientists and nutrition scientists in the world, someone was giving him a hard time and saying, “These foods are full of chemicals.” He said, “That’s right. Foods are chemicals. The point of eating foods is to get chemicals in the body to replace the chemicals you lost from the body by the normal processes of living.” We are collections of chemicals and we need to eat chemicals to put the chemicals back.

It’s not only chemicals, it’s also economics, it’s also religious, it’s also cultural, it’s also love, and prestige, and social signaling, and the environment, and sustainability, and animal welfare, et cetera. These are all involved. That, I think, often makes things very emotional. It means there are a lot of vested interests. All those makes things very challenging.

I think there are other things that are less at the emotional and social level, but just at the nature of what we do. One is that we’re dealing often with modest effects. There are some effects that are probably really big and we probably got most of those already. We don’t think about them because they seem boring and old, and we don’t bother to say them, but they’re important.

We learned, for example, that food contains energy. We learned that with Lavoisier and others a few centuries ago. We learned that you can’t live without certain vitamins, which is why they’re called vitamins. We’ve learned that you can’t live without certain minerals like salt, so important. It’s where the word salary comes from. We’ve learned those things and we take them for granted.

Most people would agree, although some people occasionally say things that make us think these days, they question that you probably can’t have too few calories or you starve to death or [00:08:00] become unhealthily thin, or you can’t have too many calories or bad things happen on the other end. We know that you don’t want certain microbes in your food, not any microbe, but certain microbes. You don’t want certain other substances and you probably don’t want a lot of mercury or lead in your food.

After that, things start to get to be much smaller effects. We might say, “Is it good to have some blueberry anthocyanins?” Maybe, probably, but it’s probably not on the order of the effect of smoking on lung cancer. “Is it good to eat a little more omega 6 than a little more omega 3 or a little more omega 3 than a little more omega 6?” These are likely to be modest effects. Now, this creates a challenge, because if we think of these effects as being long-term effects on things like longevity and heart attacks, we need very large samples in randomized trials with rigorous ways of making sure people stick to the diet for many years in order to unequivocally nail the answer down at a trial.

Well, in general, that’s not going to happen. We’re not going to get 10,000 people and randomly assign them to eat blueberries or not blueberries in sufficient quantity every day reliably for 20 years and follow them up. It’s not going to happen. That limits our ability to nail it down.

The other way, we’ll say, “Well, let’s do an observational study. Let’s study people who say they eat more or less blueberries.” Well, now we’re into self-report measurement, we’re into confounding, we’re into people not being accurate, incomplete data, all the other problems that happen with observational data, or we can do it in the mouse. Mouse is not a person. When we can triangulate, when the mouse tells us the same thing that the observational study seems to tell us and the same thing that the short-term, small trial on a [00:10:00] biomarker seems to tell us, then we start to have some confident belief.

When they don’t all line up, and they often don’t, it gets hard to know what to conclude. That’s part of the problem. Whereas looking at a vaccine or a pill, a drug for certain things as bigger effects, often we have much more ability to just nail it down. We may have to accept some limits of our knowledge. That’s especially true if the delivery mechanism is not a pill, but the delivery mechanism is food.

If you say to me, “What’s the effect of taking this drug that the FDA might approve on this health outcome?” I can pretty much assume that every pill’s going to be almost exactly the same as every other pill within some tiny, tiny manufacturing tolerance, but if you say to me, “What’s the effect of eating blueberries or eggs,” or whatever, those are not manufactured in a laboratory, they’re manufactured on bushes and in chickens. Those don’t have the same tightness of tolerance. This is not the same degree of precision engineering.

The chicken egg that I eat in Indiana in this season when the chicken was fed this way is a different chicken egg than you might eat in Denmark if you’re visiting in a different season, raised a different way, and so on for the blueberries. If you say to me, “We’re not talking about just pills, we’re talking about foods.” That makes it even harder still. Let’s make it one level harder for nutrition. Suppose you say to me, “What we’ve learned now is the whole matrix of the food matters.” People traditionally or in the last few decades, maybe the last half of the 20th century, thought, “Cheese, oh, that’s a really unhealthy food because it’s got a lot of sodium usually, and it’s got a lot of saturated fat usually, so [00:12:00] bad.”

Think Michael Jacobson from CSPI coming out holding up a plate of fettuccini in the early 1990s on camera and saying, “This is a heart attack on a plate.” Then people came along, particularly Arne Astrup from Denmark and said, “Wait a minute, cheese is not sodium and saturated fat, cheese is cheese. It includes sodium and saturated fat, also includes protein, calcium, and many, many other substances.” We have to look at the whole combination because the matrix, the whole combination of them may impart an effect.

He did a study that involved randomizing people to lots of cheese or diets containing lots of cheese and found it didn’t seem so bad, maybe even seemed good. It didn’t give negative effects on certain outcomes you might have predicted from sodium and saturated fat. He said, “It’s the matrix.” [00:12:55] “Fine, I accept that. It’s the matrix.” You can’t just predict it by having me add up sodium, saturated fat, protein, calcium, you have to actually test the food. If you say, “I’ll accept that premise,” now is Camembert the same as American cheese, the same as Gouda, the same as Swiss, Feta, et cetera? No.

If you tell me Arne Astrup did a study of cheese, I say, “Which cheeses? Am I going to get the same effect with the cheeses I eat?” Even is one batch of Gouda and Feta and Swiss the same as the next batch? I may not be able to generalize well. These are some of the many challenges we have with nutrition.

Melissa: Just dealing with food versus a pill. Absolutely. When we’re looking at research design, the execution of studies, obviously analysis of data, the statistics, some of the questions that come to mind are, are we asking the right questions in nutrition and obesity research? You already [00:14:00] mentioned epidemiology or observational studies versus randomized control trials. That’s huge differences in, obviously, correlation versus cause and effect, and the self-reporting in some of the observational studies. Obviously, there’s limitations in that.

Are we asking the right questions? What about the data analysis? Then we’ll get into also the communication aspect, the translating the science into sound bites, if you will. If you could just address some of those areas as we’re moving forward here.

David: I think the answer to the question of are we asking the right questions is that’s a matter of opinion, right is who’s judgment about right. I think we can make a very good argument that we are often asking questions that are very important and relevant and should be asked. Then we are often asking questions that we are foolish to ask either because they’ve been asked and answered sufficiently already, or we’re not trying to answer them in a way that’s likely to actually answer them to any degree of satisfaction, or they perhaps are even inherently unanswerable. I think all of these things are true.

Some studies we do I think are important and interesting and beautiful. Science can be done for lots of reasons. It can be done because it’s important, it’s applied or figured out. It can be interesting, intriguing, it’ll lead somewhere in the future. Sometimes it’s just beautiful and it helps us understand things, and eventually it leads to something, but we can’t predict that so well in advance.

There’s a lot of science that’s done for reasons that are different than that. Some of that we probably ought to reconsider a little bit. One type I would call repetitive non-probative research. Krista Casazza and I published a paper on the need for probative research in obesity-related research to make more progress. [00:16:00] An example of non-probative– By probative we mean research that really moves the needle of knowledge forward. Again, that’s a little subjective, but we think often you know it when you see it. Let’s just say that there are a hundred studies out there that have shown that regularly skipping breakfast versus regularly eating breakfast is associated with more obesity.

By the way, that’s not fanciful. There are probably more than a hundred studies at this point of that. We’ve reviewed them a little bit, Andrew Brown and myself and others. It’s unequivocal. Skipping breakfast is associated with more overweight or obesity than is regularly eating breakfast. There’s no question about it at this point. Asked and answered, move on.

But people keep doing those studies. What they do, if you might say at the very best, maybe they’re helping a graduate student get trained with a practice project or build their CV a little. Maybe it has that small value. For knowledge, zero. What’s needed are randomized controlled trials. Say, the first observational epidemiologic study, fine. Let’s take a look, let’s get an idea of what might be out there. After a few when you know there’s an association, it’s time to do the randomized controlled trial.

Now we and others have actually done them, and people can argue about whether we did the best ones or the perfect ones or tested all the right things. There are enough out there which seem to suggest that, at least under the circumstances studied, eating versus skipping breakfast regularly does not lead to better weight or obesity outcomes. You might as well skip or eat depending on which is easiest for you or most pleasing to you from the point of view of weight or health or weight or obesity. There might be other health aspects that have a different thing. That’s that idea of not doing the right study.

Another observational epidemiologic study is just wasting our journal pages, it’s wasting our money. [00:18:00] We could put that money toward better studies. And, importantly, what we know from social psychology is it increases the belief in something. It becomes epidemiology as advertising as opposed to epidemiology as advancement of knowledge. It advertises the idea and it gets it more and more ingrained in people’s heads, but it doesn’t actually advance knowledge. We increase belief without increasing knowledge. That’s a big waste of time. We do other studies that are just plain foolish.

We’ve pointed out some studies where there have been studies showing that children with bigger pants sizes, who buy bigger pants have bigger BMIs. Really? Was that a question? Did we need to know the answer to that? There are some other things like that.

There was one a little bit tangential from obesity and nutrition of somebody showing that– this research has now actually come into question, as I understand it, but showing that drivers were more likely to stop for female hitchhikers who had larger busts than female hitchhikers with smaller busts. You say, “Really? Was that an important study? Was that something we needed to spend our research dollars and time on, or a study of extra oxygen from a company marketing oxygen for athletic performance at sea level?” You say, “What reason would you have for thinking that’s a good thing?”

There’s a lot of stuff we do like that where it’s a waste of time. Particularly for those of us in academia, I think there’s an ethical imperative to not waste students’ time doing silly projects that don’t advance their career, to not waste the taxpayer money that pays my salary and so many other people’s salary on foolish research, to not get the public’s attention drawn to things we already have figured out instead of really advancing knowledge. We do too much of that.

We also spend [00:20:00] money on things sometimes where you say, “This is never going to lead anywhere,” and again, that’s often the third and fourth epidemiologic study when we’ve already had one or two instead of the randomized trial, where we may need the better epidemiologic study, even if we can’t do the randomized trial. How about not just any old epidemiologic study? How about a special one?

If you do a study showing that breakfast consumption is associated with less obesity in a general population and your only method of controlling for confounding is asking people their socioeconomic or their education and calling it socioeconomic status and saying you control for it, that’s pretty weak. How about after you do that one, instead of my doing the same thing, how about instead I do a co-twin control design, where I look at monozygotic twins, identical twins, where one eats breakfast and the other doesn’t? Now I’ve controlled for genetics and household environment if they grew up together.

Or how about we look at a school program in which breakfast was introduced in one school and not in another? It’s not a pure randomized trial, but at least it’s a quasi-experiment, et cetera. There are many others I could describe, but those are examples of things we can do that go beyond the ordinary. We do too many ordinary observational epidemiologic studies when they’re no longer useful. We do too many weak studies. We don’t need a lot more studies looking at satiety after eating this versus that when we already have those and then making claims about obesity. Do you want to make a claim about obesity? Study obesity, don’t study how hungry people tell you they are.

Melissa: Wow. [chuckles] That’s a lot. I am familiar with Andrew Brown’s research that I hadn’t realized until recently that you were working with him, so that’s very interesting.

David: I’m very proud. Andrew’s one of my– he’s now my colleague, but he’s my former mentoree and he’s been a great success.

Melissa: I remember I spoke with him a while ago, so tell him I said hi. I [00:22:00] was trying to get him on the podcast, but I was having a lot of technical issues, but the research that you were discussing about breakfast, I just thought was so fascinating. To that point, I just spoke to a group of fourth-grade students today and I just had to talk to them about correlation versus causation because the great thing is they’re learning critical thinking already.

It’s just so wonderful to see– I said, “This is the most important thing you need to know, is this critical thinking.” We talked about correlation versus causation and I feel like– I wonder, especially when it comes to research, nutrition research in particular, if that’s the biggest point of confusion for the layperson and maybe even some professionals. That’s why I like to talk about it a lot on the podcast, is if we can just remember those differences, that association, breakfast and weight or diet soda and weight, it’s like, “Okay, there’s a correlation.” It doesn’t mean one caused the other and so on. I’ll get off my soapbox.

Building on what you’ve talked about, is there anything– We’ve addressed the study design, and really appreciate your insight on replication is one thing, but let’s not keep repeating something that doesn’t need to be repeated. What about data analysis? Again, you’re a statistician, what can you share with us from that standpoint that would be helpful for us to know, and then also the communication aspect?

David: Sure. I think it’s important to separate the idea of what the general public knows and needs to know and wants to know and what a practicing scientist in a particular area knows and needs to know and want to know. I think that’s an important distinction, and particularly when we think about some of the questions you asked about statistics and correlation versus causation.

I think most educated and intelligent people, even who are not academics and professional scientists, get that correlation is not causation. They may not always remember it, but they get it. What I think is not always gotten out to the [00:24:00] public and I think sometimes is disregarded or glossed over even in academia, sometimes intentionally, and that’s disingenuous, sometimes unintentionally and naively, is that even if you say, “I’ve controlled for some variables in my analysis,” it doesn’t mean you made confounding go away. It doesn’t mean that correlation is now causation.

There are some very special circumstances under which it does, but they’re very theoretical things that I can specify in a book. I can say, “If the following conditions occur, then controlling for a variable in a statistical analysis takes care of it.” For example, if I said, “If I’m looking at the effect of a variable X1 on an outcome Y,” and you say, “I’m worried it could be confounded by X2,” and I say, “Okay, let’s assume that X2 may or may not affect Y, but it doesn’t do it through X1 and X1 doesn’t do it through X2, and there’s no measurement error, and I’ve got a random sample,” and I can go on with a few other things, then it’s true that if I control for X2, the hypothetical confounder, the correlation between X and Y now does indicate causation.

Those are a strict set of assumptions and there are yet more assumptions I didn’t bother to specify, like Y doesn’t cause X1? If I say breakfast consumption associated with obesity, maybe obesity causes breakfast consumption or lack thereof. I have to make lots and lots of assumptions. They’re a little bit heroic. If I make them– Yes, controlling for variable does it, but when you say to me, “Did you control for socioeconomic status, or did you control for smoking, or did you control for the fact that maybe obese people don’t want to be obese and that’s why they skip breakfast?”

Then I say, “I asked them that, and all those [00:26:00] variables have measurement error, and my sample is not a random sample. There are other possible confounders that I can’t specify because I don’t know all the confounders, and my variables are measured with error,” et cetera, then all bets are off. We’ve shown, even in a mouse study, where we’ve done what you could imagine the most rigorous study imaginable, suppose you said to me, “I want to know what’s the effect of energy intake, that is calorie intake, on longevity, how long you live?”

And I say, “Okay, well let’s go out and ask people how much they eat every day. We’ll do some questionnaire and then we’ll calculate the calorie intake and we’ll correlate that with how long they live.” You say, “We can do that, David, but could be measurement error, it could be they lie when they say how much they eat, they’re mistaken, it could be that’s confounded by smoking. Maybe smoking is an appetite suppressant, so smokers eat less, it could be that richer people eat at restaurants more, but you eat more when you eat at a restaurant. Socioeconomic status is involved, education.”

I say, “Yes, those are all good points. Could be genetic factors. Some people with certain genotypes or from certain ethnic or geographic backgrounds may eat more or less than other people and live longer or less.” You say, “Hey, those are all good points.” Somebody comes and say, “Okay, I’ll tell you what, we’ll do this only in people from Finland who have lived there for many, many generations. They’re a little more genetically homogeneous. We can forget about confounding by genetics and we’ll ask them how much they smoke and we’ll control for that. We got rid of the confounding by smoking, and we’ll ask them their education and their income. That takes care of socioeconomic status,” and so on.

They say, “Now I did all that and now I still got the same result, so that means I control for–” “You don’t have to worry about all that confounding,” to which I would apply even before I do any observation. Nonsense.

Do you really mean to tell me that if you [00:28:00] and I have the same income, same salary, but you are married to Bill Gates and I’m not, that you and I have the same socioeconomic status? Do you really mean to tell me that just because you and I have the same degree of education, we have the same socioeconomic status? Do you really mean to tell me that if you and I are both of European ancestry, that there’s no genetic differences among us? That if you say you smoked a little and I say I smoked a little, that means we smoked the same amount. All that’s nonsense. It also, again, doesn’t take to count all the factors that we can’t think of.

We did a study, I said, “What if we tightened it up? What if we had subjects and they were all genetically identical? They weren’t just similar, they weren’t all just from Europe or Finland, they were genetically identical. They were like identical twins.” “Okay, good, let’s do that study.” Then I said, “Furthermore, let’s never have anybody go to a restaurant. Let’s not ask them what they eat, let’s measure what they eat and let’s give them all the same food so we know exactly what they eat, and let’s have all of them never smoke. Let’s have all of them live in the exact same houses.”

A human, you just can’t do that, obviously. Guess what? In mice, that’s what we do. We use inbred mice, and because of the inbreeding, they’re genetically identical within probability. Then we feed them whatever we want to feed them. There’s no smoking and there’s no restaurants and they’re all in the same cage and they’re all the same, I guess, socioeconomic status. We do that and there’s no missing data and so on.

We’ve done that study, actually. We randomly assign mice to eat different calories. Then we have another part of it where we just let them choose to eat different amounts. Guess what? This is a better controlled study than anybody will ever do in a human. We get the opposite answers. The correlation of how much the mice choose to eat with longevity is positive. The more [00:30:00] calories the mice choose to eat, the longer they live, but the causal effect of us assigning them to eat more calories with longevity is negative.

The more calories we give them to eat, the shorter they live. That shows you that what’s probably happening is self-selected calorie intake or energy intake is being influenced by health (healthier mice probably like to eat more), which then is also influencing longevity and confounding it. What this says, at least if you’re this kind of mouse, is if you feel like eating more, you’re probably healthier and will live longer, but if you actually eat more, then you will probably live shorter than if you actually ate less.

We can figure that out on the mouse, but what that tells me is there is no amount of rigorous measurement and controlling for confounding in these conventional ways in humans that will be guaranteed to give us the right answer. It only gives us a closer look. We need to be open about that, and many of the people in the nutrition epidemiology field won’t accept that. They will offer disingenuous statements, such as, “Well, measurement error only leads to associations going away, not being present,” which is completely untrue. We know theoretically it’s untrue, it’s been shown empirically to be untrue over and over.

There are many, many factors that can lead to biases, toward the null hypothesis, meaning toward not finding an effect, or away from it, meaning finding an apparent effect when there is no effect. We have to accept that. That doesn’t mean we shouldn’t do observational epidemiology, but we need to interpret it, if you’ll pardon the pun, with a substantial grain of salt and not think just controlling for factors takes care of it.

Melissa: Wow. That’s a lot. I do need to say [00:32:00] that I got an A in all three of my statistics classes. I’m just trying to follow, but it’s all very interesting, and again, speaks to the nature of nutrition research, the challenges and the limitations. Let’s get a little bit more specific. There’s been a lot of debate in the field the last few years about macronutrient composition or other aspects of the composition of the diet beyond energy content, so calories, and the effect on body weight and body composition change. We hear a calorie is just a calorie. What have you seen in the research? What’s your take on even just the nature of that question and what the evidence shows?

David: I think this is one of those areas where it’s fraught with both unclear communication, with emotional kind of dialogue, competitiveness, pandering, and just sloppy thinking. One of the challenges is that people are often mixing up lots of different questions, and so some of the debates are really pseudo debates because people are talking past each other.

An example would be this, let’s just say that Chris Gardner, and Chris has said this, he’s a scientist who studies diets and macronutrient composition in randomized controlled trials, does a trial comparing two or more diets that differ in macronutrient composition. He assigns the diets to people and gives them instructions on how to consume them. Then he asks them over time, “What are you consuming?” He finds if he waits long enough, there’s no major difference among the diets in terms of the outcomes they have on weight, or that one diet looks a little better than the other.

Then people come along and say, “Chris, the people in your study are not really eating what you told them to eat, so you weren’t really studying diets A, B, C, and D, you were studying something that vaguely was intended [00:34:00] to be diets A, B, C, and D. In fact, Chris, if you look at diet B, it turns out that what people are eating when they’re assigned to diet B is actually what I, the progenitor of diet A, would recommend people eat in diet A. What you’re calling diet B is really my diet A, and when it looks better, you’re showing my diet to be better.”

What Chris might correctly respond to them in that situation is, “I wasn’t studying the molecules of the food, I was studying the instruction to eat the food. What I study in a randomized trial is treatment assignment. I study the effect of being assigned to eat diet A, B, C, or D, as an example, not the effect of actually eating diet A, B, C, or D.” Someone else could say, “Well, I don’t care about the effect of telling people, I want to know what the effect of actually eating diet A, B, C, or D is.”

Well, if you want to know that, we probably need to do a controlled feeding study, like David Ludwig from Children’s Hospital, Harvard University, and I have just finished, in which we brought people to the tune of many, many millions of dollars, funded by Arnold Ventures, to consume different diets, and they consumed all their food under strict supervision. They were in a residential facility, which is why it cost many millions of dollars. Kevin Hall has advocated these domiciled studies.

If you want to study the effect of actually consuming foods, you need to do that. Even there, you might argue, “What about expectations? If I keep hearing that diet A has a different effect than diet B, that low carb is better or worse, and I know I’m eating low carb, might that affect my attitudes, my exercise behaviors, my happiness, any other thing that might somehow also have an effect?”

Ideally, you’d want to grind up those molecules into a homogenate and then put [00:36:00] them down my throat in an opaque tube, and do that for six months, but that’s hard to do. Even the domiciled study, the residential study that we’ve done, you still don’t get rid of placebo or expectancy effects maybe, perceptual effects. These are the different challenges. Are we looking at the effects of knowing you’re eating the food, actually eating the food, being told to eat the food? A lot of these are different. Are we being asked to look at the effects of the macronutrient composition ad libitum or not ad libitum?

We published a review on dietary fat. We said, “Is dietary fat fattening?” What do you mean by that? Do you mean if I eat a diet that’s higher rather than lower in dietary fat, but my calorie intake is fixed, so I eat X calories on a low fat diet versus X calories on a high fat diet, but I have no choice about the number of calories I eat, that’s one question, or are you asking if I eat a diet ad libitum? I can eat as much or little as I want, but I’m locked into the low fat condition, you hand me all the food, versus the high fat condition, let’s say, you hand me just one food, I don’t get to choose I’ll have a little more food, a little more ice cream, and a little less broccoli from the broccoli and ice cream you give me.”

You say, “No, I’m just going to give you this homogenized food. Have as much or as little as you want, but that’s all you get,” and I’m locked in, I can’t leave the room. Well, now that’s calories are ad lib, but the food composition is locked in. Or are you asking me if I eat some low fat and some high fat, that’s strictly locked, but then the rest of my diet I’m living out in the real world and I could choose to eat or not eat other things? How does that affect it? Or are you asking about a preload? If I eat a little high fat food and then you let me eat more ad lib versus a little low fat food and you let me eat more ad lib, what are those? These are all different questions.

When we naively say, “Is a [00:38:00] higher fat, or a higher carb, or a lower fat, lower carb more or less fattening?” Then we say, “Which of those many different things did you mean?” People are just arguing past each other. The second thing is for what? We often hear people ask questions like, “Which is a better diet?” To which I say, “That’s a completely non-scientific question. If you say which diet, as long as we specify what we mean by diet, promotes greater or lesser weight loss, that’s a scientific question we can answer.

Better? Better has no meaning scientifically. You have to tell me what you mean by better. You might say, “Better for me is cosmetic. I want to look better.” Someone else says, “Better for me is less diabetes,” someone else says, “Better for me is more physical strength.” What’s better? We have to look at the outcome. We tend to get into these religious beliefs, again, this all or none thinking, where I say, “Aha, higher protein is better, lower fat is better, lower carb is better.” Well, it may depend on what you want.

Maybe if you want endurance performance or sprinting performance, maybe some more carbohydrate is better. Maybe if you want less risk of diabetes, less carbohydrate is better. Maybe if you want to build muscle strength, more protein is better. Maybe if you want to live longer, less calories are better.

Melissa: Well, that sounds like we’re transitioning into, given what you know about the evidence for obesity treatment, and prevention we should mention, and let’s say for adults, how do you think we should prioritize next steps or research to enhance this treatment and prevention? What have you seen? Tell us what’s working, what’s not, [chuckles] and what next steps would look like?

David: I get asked this a lot, particularly by state government people, or foundation people, or major healthcare providers who say, “I want to do something good, what should we do?” [00:40:00] and I say, “Well, let’s take a step back. What are you really trying to accomplish?” I particularly get asked this often in the childhood obesity space around community. A foundation says, “What do we do?” Or the governor says, “What do we do?” Or the state major healthcare provider, “What do we do?”

I say, “Well, what do you really want to accomplish?” Do you really want to advance knowledge? Do you want to figure out what works? That’s one thing. Do you want to reduce obesity itself and or suffering from obesity in the near term? That’s a different thing. Or, do you want to make people feel good, know you care, and raise consciousness? That’s another thing and those three are incredibly different.

Most people, immediately, the things they’re thinking about, especially in the childhood obesity space, start off with the things that would be well calibrated to raise consciousness, to let people know you care, to make people feel good. Those things are very valuable. If, for example, you said to me, “How about we build a walking trail near the local school and we have farmers come and maybe set up a farmer’s market with fresh fruits and vegetables in the parking lot once a month at the local schools and we do a 10 minute in-school calisthenic exercise program three days a week in the schools.”

I would say that’s probably going to really show people you care about them. That’s good. It’s going to make people feel good. That’s good. It might get some young kids in that program to say, “What’s all this obesity stuff about? Let me think about that.” The stuff they’re doing with the walking trails and the farmer’s market and the 10 minutes of exercise, this doesn’t seem like it’s maybe that great, but I’m interested in this problem and that kid may grow up [00:42:00] to be the next Einstein or Pasteur, et cetera, and figure out what you and I haven’t been able to figure out.

They’ll come up maybe with the really great program, but if you ask me, is this thing going to reduce childhood obesity levels? No. I think we have pretty good evidence that that will have somewhere between zero effect and an effect that’s so small that it’s hardly going to be worth the effort. We need to know what’s the goal.

Second way, if you said, “No, no, I really want to reduce obesity suffering, I want to reduce obesity.” What do we really have great confidence we can do? I’d say, “How much money have you got?” You got X million dollars? Good. Put that aside. With that X million dollars, you’re not going to be able to give the best treatments to everybody, but you could give it to some people.

Let’s come up with an algorithm that either figures out who will benefit the most from treatment, or let’s have a lottery. Randomly pick people to be fair, and some subset until the money runs out, who want the treatment can get it. Then I’d say bariatric surgery is the most effective health-promoting treatment we have, at least for severely obese people, including adolescents, maybe not young children.

Perhaps not everybody’s ready to go that far, but for adolescents. Or some of the new drugs are very powerful and some of them are starting to rival bariatric surgery. The SGL2 inhibitors, the GLP1 agonists, the GLP1 GIP drugs, these are all potentially very powerful and may be new drugs coming down the pike.

Then there are some much less expensive and much more mild drugs, but that have smaller effects, but still beneficial effects. We could say do that. If you said to me, “Really? Really, Professor Allison, your solution to the public health problem of obesity and the scourge of childhood obesity is surgery and drugs?”

I’d say, “Well, we have a [00:44:00] few other clinic things that are meal replacement based and just clinical family treatment, those can be effective.” They’re less effective, but they’re still effective. But what we know is really effective are surgery and drugs, clinic-based, somewhat effective, school community-based, either not effective or the ones we’ve tried so far, such small effects that’s really hard to advocate them in good conscience.

I think we have to say at that point, yes, it may not be feel good stuff, but that’s what works. We need to accept the world as it is and as the data show it to be and not as we think it should be. In the future, I hope we get to a point where we have non-surgical, non-pharmaceutical treatments that are so effective in preventing or reducing obesity with so little side effects and so inexpensively that we say, “My goodness, how did we ever live in a world where we thought surgery and drugs were the answer?”

But today, we do not live in that world. Today, we live in a world where the community-based stuff and the preventive stuff has not been shown to be particularly effective. We need to accept that. Then the third thing is, do you want to advance knowledge? If you said to me, “Well, maybe I won’t be effective today, but let’s figure out what might work tomorrow.”

Say, let’s study some things. Some are going to be the creative wild molecular stuff and some are going to be the social stuff. We have some pretty compelling evidence that suggests, and I don’t want to say proves, but suggests that what you might call the good life leads to less obesity.

What do I mean by the good life? I mean, parent training, good parenting, a safe neighborhood, not being at the absolute bottom of the social totem pole, general education, and I want to underline that word general, general education, not nutrition education, seems to lead people to better health outcomes and less diabetes and less obesity.

There are even some randomized trials suggesting it, and this seems to be [00:46:00] particularly true for women and girls. There have been a few studies that have provided some of these elements that I’ve called the good life that I just described to you to people early in life and then followed them 30 years later. These were randomized experiments and found less diabetes in obesity, particularly in women.

I think those were serendipitous findings. Now we need to call the shot in advance. Those are going to be many tens of millions of dollar studies to repeat but those I think would be the big, big studies that an NIH, a Department of Housing and Urban Development, a Arnold Ventures, a Gates Foundation could fund and we could figure out that answer. Wouldn’t it be wonderful if giving people a better life also happened to be the public health approach to preventing obesity and diabetes?

Melissa: Wow. Well, I can certainly see how your education and experience earlier in your career in psychology has influenced and informed your work in obesity-related research. As we’re wrapping up, is there anything else you wanted to share with us about this topic? Also, just a really short question, I think, your opinion on research that doesn’t get published. Any great advice on that limitation or that challenge and how we can work with that situation?

David: Yes, ma’am. I am so glad you asked that question because this is a vital need. We often hear people within the scientific community and some beyond it saying they’re concerned about trust in science and we need to promote trust in science. I think science is not about trust. Science is about method. We should trust the findings of science to the extent that the methods are trustworthy.

We should be promoting trustworthiness in science and then asking people to trust our science because it is worthy of trust. Not just saying [00:48:00] blanket trust science. What we know is that most people in the United States do trust science. Trust in science is down a little but not a lot, and it’s down less than trust in most other things.

Trust in government is down, trust in industry is down, trust in most things are down, trust in science is down, but less than in those other areas. But when you look at trust in science, there’s the Pew Charitable Trust actually, it looks like trust in nutrition science is much lower than trust in other science.

Trust in nutrition scientists is lower than trust in practicing dietitians and medical and health providers who provide information on nutrition. The public has learned to be a little skeptical of us. Why? Because in fact, I think what they have realized is in part we have met the enemy and it is us. That as we in nutrition science over hype our results often.

We talk about things prematurely. We’re not always good in the general public about distinguishing between correlation and causation. We’re not always honest about measurement error. We exaggerate results from mouse studies and not point that out. Those are many things. And the childhood obesity literature seems especially wrought.

It seems that that self-righteousness of– my heart goes out to children, we must protect children, leads us to be willing to bend the truth. I think we need to stop. We need to say not as a scientist, there’s no bending allowed. That’s part of our identity as a scientist. Unequivocal, unshakeable commitment to the unvarnished, truthful presentation of results.

Some of that means including all the results. If you go to Washington, DC and you go to the Keck Building of the National Academy of Sciences, emblazoned on the wall on the outside, you walk past it as you walk by, is etched into the stone so large that you can’t miss it a quotation from Einstein and he says, [00:50:00] “The ability to work in science comes with it an obligation, a moral commitment to tell all of the truth, not just some of the truth.”

All of the truth means, publish all your results, not just the ones you like. We need to shore that up. It means not doing a study in children, that was meant to look at effects on weight as the primary outcome and not getting a significant result and then publishing and saying, “Look at the great results we got on fruit and vegetable intake. This is a good thing for obesity.”

It means not hiding the fact that you said, “I’m going to look at girls and boys together as the primary thing,” and you have no prior hypothesis that it’s only going to show up in girls or only show up in boys. When you don’t get a significant result in girls and boys together, you say, “Aha, but I got it in one of the genders. It looks like it really works for them.”

I could go on and on about saying, “You said obesity, what you showed was a half-a-pound difference in weight and nobody went from obese to not obese.” These are some of the many things. We have a whole paper on this about how childhood obesity literature’s exaggerated. One of the worst offenders is the cluster randomized trial.

This is where you assign clusters like schools or classrooms or clinics or families to treatments rather than individuals, but then you analyze the data as though you had assigned the individuals. I won’t get into the statistics because it gets complex, but any statistician worth his or her salt will tell you, that is completely and unequivocally invalid. Yet, it is done over and over and over and leads to claims that a finding of efficacy has been obtained when a finding of efficacy has not been obtained by a reasonable statistical analysis.

My group and I point this out often. Many papers have been retracted as a result of us pointing this out, but the naïve reader [00:52:00] of the childhood obesity literature would think there’s a lot more efficacy demonstrated than has been demonstrated. We need to look at the man in the mirror and the woman in the mirror and say, “If we are the academic and scientific communities, we need to hold ourselves to a standard of truthfulness that is greater than we are doing today.”

Melissa: All right. Wow. Just a quick follow-up to that, because we know that the public wants answers. They don’t like gray areas, they don’t want those little modest changes. They want big information. They want it to be earth-shattering. When it comes to the communication aspect of that, anything more you can say other than we need to fix this, we need to change this, we need to tell the whole story? Anything with regards to how the people will respond to [laughs] what we’re trying to do better communication?

David: Sure. I think one thing is to accept that we as the communicator can’t always have everything we want. Sometimes we should do the right thing. Not sometimes, always we should do the right thing because it is the right thing. This is one of my favorite quotations and it actually comes from one of my favorite sources of wisdom, which is Batman.

Batman says to Robin, “We do the right thing because it is the right thing and no other justification is needed.” Some people say, “But if I do the right thing, I communicate in a way that’s honest, people won’t pay attention to me.” To which I replied, “Then accept that.” Now it may not be true, it may be that sometimes you can say the right thing and people will pay attention and we can come back to that.

But if that means not enough people paying attention, so be it. There are times when I default quite intentionally when speaking with journalists to being very eggheady. I know that sometimes I talk about eggheady [00:54:00] things like measurement error and residual confounding. There’s a good chance that they’re not going to use my comment and I’m okay with that because the NIH doesn’t fund me to make David Allison famous. The NIH and others fund me to pursue truth through science and communicate truthfully. That’s important.

The second thing I would say is not everything needs to be communicated. Again, the purpose of science is not to make me or anybody else in academia or science famous. It’s to advance knowledge. Maybe not every mouse study I do needs to be in the public eye. Third thing I’ll say is, science is beautiful. I love science. There’s a lot of times when I read, and I’m not a mathematician, but I’m a mathophile  [00:54:44].

A lot of times I read to the best of my ability to understand some new finding in math or in physics. I’m not a physicist either. I love it. It’s so cool. I’m not doing that so I can design my own space shuttle and go into space tomorrow. I do it because I appreciate the beauty of math and physics and I try to think about it and that’s okay.

I think we should present our science in nutrition and obesity and metabolism that way sometimes too to say, “Isn’t this cool? We’ve now understood this feedback loop that we never understood before. We’ve understood that what we thought was going on in evolutionary biology that led to this aspect of physiology metabolism is different than what it really is.” That’s cool. That’s interesting. This marmoset study is fascinating. This drosophila study is so interesting. That doesn’t mean that you need to go out and change your diet tomorrow based on this.

Melissa: As you’re talking, I’m realizing, “Wow, maybe we’ve had it all backwards.” We think that we won’t gain the trust if we don’t give them something more concrete, [00:56:00] when in fact it’s the other way around. We can gain trust by trying to articulate those nuances and the challenges. Very good.

Was there anything else you wanted to share? I know I’ve kept you on for quite some time and you’re very, very busy. I know I have links to your LinkedIn profile and your profile at the university as well as maybe you could speak briefly to the Obesity and Energetics Offerings. I have a link to that. That has been in my Sound Science Toolkit for my audiences since 2018. Maybe you could speak to that. Anything else you wanted to share as we’re wrapping up?

David: With respect to the Obesity and Energetics Offerings, thank you for mentioning that. The Obesity and Energetics Offerings are something that a team of us, which include Andrew Brown, Colby Vorland, Greg Pavela, myself, and many others who I apologize in advance to them, I won’t mention all their names, but they’re on the website. It’s a free service.

Anybody can sign up for it. You just type in Obesity and Energetics Offerings into Google or whatever your search engine is. It’ll find it for you. Then you just click a bunch of things, send your email in and you’ll get it every Friday. We’ve done it for well over a decade. We have never missed a Friday. Not on New Year’s Day, not on Christmas Day, not on Thanksgiving, not in snowstorms, not in pandemics, we never miss.

We get it out and it contains usually about 100 links to scholarly, mostly elements within the topics of obesity, energetics, nutrition. People seem to love it. We don’t give a lot of commentary about it. We just say, “Here are these links.” I encourage people to sign up. We don’t accept any commercial funding for it. We just fund it out of our non-commercial sources.

The other thing I would say is science is both a great tool and it’s a great fun. I would encourage as many people as want to join us in doing science. If people want to join us right here at IU and come join us as [00:58:00] students and postdocs and faculty, I hope they’ll reach out. If they want to collaborate with us, I hope they’ll reach out. If they want to go join another university and do science or do citizen science or just learn about science so they can be a better consumer of it, I think that’s all great and hope they do it.

It’s both beautiful, fascinating, wonderful, and interesting. It’s also important. We recently published a paper on lactation cookies and this came about at the beginning of the pandemic when I thought, “Maybe I’ll get some cookies for my family and have them in the house while we’re all locked down.”

I went to an online store to get some, typed-in cookies, and up came something called lactation cookies, which I had never seen before, and had a price tag that was on the order of six or more times the price of typical cookies. I thought, “What is a lactation cookie?” Well, it’s a cookie that’s meant to increase lactation among women, which would then help them breastfeed their offspring better.

I know that this is from other research we’ve done that the challenges of breastfeeding for some women is a source of great stress and the social stigma of not breastfeeding is also a source of great stress. Women are often in this double bind. They’re very vulnerable. Not every woman could throw away large amounts of money on cookies willy-nilly. If they didn’t do that and the cookies didn’t work, they could maybe try something else.

Furthermore, if they’re eating a lot of cookies that have little to no effect, then they’re also probably getting a lot of calories and maybe not the healthiest diet. We looked into this and I couldn’t find any evidence that showed that lactation cookies worked. I could find lots of reasons to believe they could work, but that’s not that they do work.

We have a saying in our group called, it’s about knowing and there’s a book we’ve put out about that. By the way, if everybody wants it, send me an email, we’ll send you a free copy of the book. We say it’s about knowing because conjecture is good, but knowing is better. In this case, knowing is important, not just because science is fun and interesting and beautiful, knowing is important because real humans [01:00:00] are involved. The health and quality of life and well-being and economic standing and stress of women and their children are at stake. We did a randomized trial. We fund it ourselves out of pocket. Ana Palacios led the trial with myself, me as the senior investigator, and others involved. We showed for a particular type of lactation cookie, no effect whatsoever on milk production.

Now that doesn’t mean that no lactation cookie under any circumstance in any dose for any woman could ever have any effect, but the kind we used in the way we used it, and the dose we used, it had no apparent effect. I think this shows the importance of really asking rigorously, what do we know from science?

Melissa: Absolutely. Well, thank you so much for sharing all of this information and your insights and just the amazing amount of work that you do. I’ll continue to follow you. Like I said, I’ll have all the links to the resources that we discussed in the show notes, including to the Obesity and Energetics Offerings. Although, like you said, people can just Google it.

I’ll also have a link to my Sound Science Toolkit so people can sign up for that. Because to your point, if we’re going to be better consumers of nutrition research, I have curated a ton of resources for people to do that and to remind us what the difference is between an RCT and an observational study is, and much, much more. That will be there as well. Thank you again, and for everybody listening, as always, enjoy your food with health in mind. Till next time.

[music]

Voice-over: For more information, visit soundbitesrd.com. Music by Dave Birk, produced by JAG in Detroit Podcasts.

[music]

[01:01:55] [END OF AUDIO]