Ontario must be a strange place. Aquarians are 23% more likely to experience chest pain than all other signs of the Zodiac. Their hospitals admit 13% more people with the sign of Pisces for heart failure. Librans suffer from 37% more fractures of the pelvis than other patients. Or so we learn from a publication (1) described in Ingredients by George Zaidan. No, our Zodiac signs don’t condemn us to greater incidents of specific medical ailments. Mining for significant relationships in large databases may lead us astray. Are the relationships data miners find real or due to Random Chance?
Do the revelations about Zodiac signs and disease discredit studies about ultra-processed food? No. They do call into question the whole process of mining of large databases. No rational hypothesis exists that Geminis exhibit higher rates of alcohol dependence syndrome than Aquarians. Do diets high in sugar, fat, and salt increase chances of chronic diseases? Mining this dataset based on a stated hypothesis for a specific disease makes sense. The studies group together all packaged products with five or more ingredients. They call this category ultra-processed foods (UPFs).
Did the investigators state a specific hypothesis before starting the study? Or did they approach the dataset with the idea to run every analysis they could think of? Did they only report the ones that turned up statistically significant? Nothing we know to this point clarifies the situation. If they started with a hypothesis and tested it, that is a valid course of study. If they only sought significant, publishable results, they committed p-hacking, a questionable scientific practice. Who cares? Their work becomes equal to the Zodiac studies. Such research demonizes otherwise healthy foods without valid scientific evidence. How do we know which UPFs are unhealthy? Which UPFs may even promote good health? For example a sample hypothesis could read:
Consumption of large amounts of ultra-processed foods increases levels of heart attacks?
Zaidan and I do not see that the researchers followed this path.
What is so bad about p-hacking if it helps us eat healthier? P-hacking places the emphasis on publication and publicity not on accuracy and usefulness. How do we know if the conclusions are legitimate? Important decisions about what to eat hang in the balance. Web stories gain credibility based on clicks rather than the quality of their data. The reputations of scientists rise based on citations not merit. We become pawns in social experiments based on hype not based on science. Zaidan suspects the conclusions because the researchers didn’t pre-register their hypotheses. Data miners hit paydirt, but can they distinguish gold from fool’s gold? If it’s significant, it’s published, and it’s accepted as truth. If no relationships are significant, we never hear about them.
Ultra-processed foods represent a wide range of foods up to 61% of the American diet. Yes, they include cakes, cookies, ice cream, sodas, and candy. They also include whole-grain cereals, plant-based meat substitutes, hard liquor, gluten-free bread, and lactose-free milk. Are some of these foods healthier than others in the category or are all UPFs equally unhealthy? Is the five-ingredient rule valid or simplistic? Based on the evidence he reviewed, Zaidan rejects the conclusion that UPFs are dangerous.
Didn’t the NIH study prove that ultra-processed foods produce weight gain? Data enlighten. They do not prove! Zaidan commends the NIH study as the first randomized control study on the topic. Kevin Hall, the leader of the study, excelled in designing it. He preregistered his hypotheses and experimental design. His data analysis showed no p-hacking. These types of experiments challenge researchers, however, in their degree of difficulty and incredible expenses. Zaidan welcomes studies like the NIH one. Then he criticizes the conclusions more harshly than I did. He states that confining the subjects to a hospital for a month fails to approximate real life. The subjects experienced many invasive medical procedures that defy reality. The study covered too small a sample size over too short a period of time. Zaidan praises Hall for a good start. He presents no alternatives for a better study. So where does that leave us?
What can we learn from Brazil? Brazil is initiating a massive experiment on its population. Dr. Carlos Monteiro used NOVA to design a radical new set of dietary guidelines. These guidelines call for avoiding all UPFs and fast foods. They encourage home meal preparation and eating regular meals with family and friends. Officials promise better health with mass adaptation of these guidelines. Not everyone agrees. Food engineers in Brazil dispute the benefits of NOVA and the guidelines. What could possibly go wrong by adopting these guidelines? Brazilians could refuse to follow the guidelines. Most Americans pay scant attention to our dietary guidelines. Excitement about the guidelines could stimulate mass changes in the Brazilian diet. After the initial euphoria, a gradual reversion back to current dietary habits could occur.
Let’s assume that Brazil adopts the guidelines and sticks with them? First, expect a shock to the availability of food in the market. Supply chains will have difficulty keeping enough fresh foods on the shelves. Prices might skyrocket. Second, benefits in health incomes need a long time to appear. Immune systems do not improve overnight. Susceptibility to chronic disease will not vanish soon enough. Weight loss takes a long time to manifest itself. Impatient journalists may wonder why health isn’t dramatically improving. Consumers may become discouraged. Manufacturers of supplements and potions begin to hype their wares. Miracle diets start proliferating in the media. Forbidden foods tempt the weak. Could this effort collapse? Who knows? Its success depends on the will of the Brazilian people.
How can data miners provide us with a clearer picture? The mass category of UPFs makes up of over 60% of the American diet. Are all UPFs unhealthy? Are some unhealthy and others not dangerous to our health? Why won’t they break UPFs into at least four subcategories? Then perform the same tests on each of these categories AND report all the results. Based on other studies, junk foods could pose the greatest risk. Distilled spirits might show dangers as well. Convenience foods may or may not be unhealthy. Functional foods may even decrease chances of heart disease and other health hazards. Data miners could then drill down within each group to determine which products are more dangerous than others.
My subgroupings are not the only way to go. Other scientists could subdivide these products into different groups. For specific foods identified by the miners, biochemists would propose potential mechanisms. They could then test their theories and refine them. We would then have a clearer idea of which foods pose a danger and which foods do not.
Where is the mechanism? Why do we need one? As I began my research career, I designed observational experiments. My graduate student, Tsung-She Cheng, and I studied changes in fresh tomatoes during storage. Tomatoes lose quality when stored under refrigeration. We developed an objective for our study with fresh tomatoes (2):
to characterize the color development and gaseous evolution rates of tomato fruit during ripening after different lengths of chilling exposure.
I learned that grant proposals require proposing and testing a mechanism. Observing and reporting results are not good enough. Mechanisms provide biochemical explanations for changes observed in a series of experiments. I teamed up with Dr. Al Purvis. We developed a physiological model to explain chilling effects in susceptible plant tissue (3). Then his graduate student, Jim Gegogeine, tested our proposed mechanism on green bell peppers (4).
My readers may not care why tomatoes and bell peppers don’t hold up well in refrigerated storage. Understanding this mechanism, however, gives the research credibility. It also points to better management practices. The results don’t count unless the mechanism makes sense. Readers do care what foods contribute to the development of chronic diseases like diabetes. Do UPFs cause chronic diseases and lead to premature death? Are these studies valid? Or are they an example of Random Chance produced by data mining and p-hacking?
The beginnings of a mechanism emerge. A study measured telomere length in elderly Spanish citizens (5). Telomere length decreases as we age. The group who ate the most UPFs exhibited shorter telomeres than those who ate the least. This study collected real data on real people. It used a mass database, but I saw no evidence of data mining. The study measured a meaningful consequence, premature shortening of telomere length. It reports telomere length as a function of a real activity, consuming more UPFs. It does not identify which of the 50 types UPFs cause the shortening of telomere lengths. The authors mention that many of these foods are high in sugar, salt, oil, or food additives. Many UPFs are not high in sugar, salt, or oil. Almost all UPFs contain food additives. The study does not identify the types of ingredients responsible for shorter telomeres.
At this point they have the beginning of a mechanism. Now, investigators must identify a few suspect ingredients in UPFs that they suspect are unhealthy. Next, they should propose biochemical steps linking those ingredients to shorter telomere length. Then, comes a test of their proposed mechanism in an animal model or cell line. Otherwise we may be dealing with Random Chance and the Scorpio curse!
Next week: Tomatoland: From Harvest of Shame to Harvest of Hope
References:
(1) Austin, P.C., Mamdani, M.M., Juurlink, D.N. and Hux, J.E. 2006. Testing multiple statistical hypotheses resulted in spurious associations: a study of astrological signs and health. Journal of Clinical Epidemiology 59:964-969.
2) Cheng, T.-S. and Shewfelt, R.L. 1988. Effect of chilling exposure of tomatoes during subsequent ripening. Journal of Food Science 53:1160-1162.
(3) Shewfelt, R.L. and Purvis, A.C. 1995. Toward a comprehensive model for lipid-peroxidation in plant-tissue disorders. HortScience 30:213-218.
(4) Purvis, A.C., Shewfelt, R.L. and Gegogeine, J.W. 1995. Superoxide production by mitochondria isolated from green bell pepper fruit. Physiologia Plantarum 94:743-749.
(5) Alonso-Pedrero, L., Ojeda-Rodriquez, A., Martinez-Gonzalez, M.A., Zalba, G., Bes-Rastollo, M., and Marti, A. 2020. Ultra-processed food consumption and the risk of short telomeres in an elderly population of the Seguimiento Universidad de Navarra (SUN) project. American Journal of Clinical Nutrition 111:1250-1266.
In Defense of Processed Food 
Processed foods have been getting bad press because of a few products that became instantly popular (example deli meat). I believe food processing has helped in a number of ways including contributing to a balanced diet: pasteurized milk, frozen fruits and vegetables, canned fish, yogurt. This side definitely needs to be promoted.
LikeLike
Thank you.
LikeLiked by 1 person
Lots of reasons people eat what they do and reject what they don’t. Many of them have nothing to do with biology. There are popular images of what’s healthy, and that gets into the press and academia and feeds itself. Sometimes the images are consciously opposed by riskers who see need for “healthy” as weakness and submission to rules. Mostly they ignore quantity and balance. Certainly, the word “processed” is misunderstood, and counting ingredients is nutritionally silly, but it appeals to people who distrust human activity. And it’s not going to go away, as it is part of the evolving cultures.
LikeLiked by 1 person
Quantity and balance are important concepts. Too many calories whether from junk foods or superfoods can lead to unwanted weight gain. Balanced meals are preferable to overly restrictive diets.
LikeLike
Calling foods junk or healthy is avoiding responsibility for balance and quantity, our decisions. Pride and pleasure are in conflict — growing up from infancy means we learn to take pride in our ability to restrain the agents of immediate pleasure to gain the familial/social belonging that is seen as security.
We evolved needing 6000 Cal/day, so we could say we don’t eat too much, we move too little. But if we think about what that movement was all about, maybe we’re better off moving less (and eating less).
LikeLike
I have no problem calling a high energy food with minimal micronutrients a junk food. I do not believe the term should be restricted to manufactured ones. Homemade cakes and cookies should also be considered junk. As the cook indicated in her discussion on NOVA with the food scientist, “what constitutes healthy remains elusive.” Your theory on why we get fat squares with such disparate sources as Herve’ This in “Note-by-Note Cooking” and Bee Wilson in “The Way We Eat Now.” I think there is some merit to the argument, but it is much too simplistic to explain our obesity crisis.
LikeLike
Sugar is, at least as per my reading, is a Group 2 ingredient. “Processed culinary ingredients include oils, butter, lard, sugar and salt.” And I hear you. My instinct too is to think of sugar as highly processed. But here’s the difference as per my reading. First, NOVA makes more sense if you think of ingredients on a timeline. Sugar has been processed since the 17th century I think. Or maybe the 18th? Anyway, sugar has been used for a long time and is familiar. Second, the NOVA document is not against fats, salt, or sugars per se when these ingredients are used in the home for baking or cooking although moderation is recommended.
LikeLike
OK, sugar is in Group Two. To a food scientist it is the most heavily processed ingredient in food. It is also the ingredient called out most often when ultra-processed foods are condemned. Nitrates and nitrites have been used to cure meats for a very long time, but they are not authorized in Group 2. Fats, salt, and sugars appear to be OK when in a whole-food matrix or added in the home, but not when incorporated into a product in a manufacturing plant. It still seems to me to be a plot against industrial foods.
LikeLike
In Hall’s experiment, four weeks in a highly confined environment with daily medical testing going on is great to control variables but not so good at replicating real life.
LikeLike
Some random comments on my favorite blog.
The best shelf stable Marinara sauce out there in my humble opinion has 8 to 12 ingredients depending on how you count. The list reads pretty much the same as the Marinara I cook from scratch. Besides every recipe for Marinara sauce I’ve ever used exceeds 5 ingredients. So is the product processed or ultra processed? The rule is SIMPLISTIC but it has unfortunately captured the public’s perception. We need to work on more nuanced criteria.
Kevin Hall’s NIH study demonstrated that meals with similar amounts of calories, carbs, fats, sugars, salt, and fibers impacted consumption patterns differently. In other words, something other than nutrients made a difference. The folks who were offered ultra-processed meals consumed more food and gained more weight than the folks who were offered meals cooked with minimally processed ingredients. There was no significant differences reported by either group in satisfaction, hunger, fullness, pleasantness, familiarity. The mechanism is unknown. More studies required to identify the mechanism.
LikeLike
Thanks again for your comments. There is nothing nuanced in the NOVA classification of foods, particularly with reference to Group 4, ultra-processed food.
Back to Rao’s marinara sauce. The ingredients listed in parentheses are the sub-ingredients of the main ingredients. I have a feeling that Monteiro and Pollan are counting words on the ingredient statements and not the main ingredients. Your marinara sauce is obviously G4 because it violated the letter of the classification scheme by containing more than five ingredients. I would have to see the ingredient statement to determine if it violates the rules in spirit. I tried to locate the ingredient statement online, but I was unsuccessful.
On Kevin Hall’s experiment, he did not exactly present his subjects with meals. He presented them with a selection of foods amounting to over 5000 calories per session to choose what they ate. The UPF subjects ate approximately 250 calories each day more than they needed to consume to maintain weight. At the end of two weeks they gained 2 pounds. The ‘unprocessed’ group ate about 250 calories each day less than they needed to consume to maintain weight. At the end of 2 weeks they lost two pounds. If the groups had done the same experiment over the course of a year, each group could have stabilized their weight or the UPF group could have gained 52 pounds and the ‘unprocessed’ group could have lost 53 pounds. As a dietitian, not a cook, if the groups were not trying to lose or gain weight, would the ‘unprocessed’ group be considered healthier than the processed group? It gets back to the American idea that any diet that results in weight loss is healthy and any diet that results in weight gain is unhealthy. From the news stories I read at the time, change in weight was the primary takeaway of the study. From that it was concluded that UPFs were unhealthy and unprocessed food was healthy.
LikeLike
Counting ingredients assumes all ingredients equivalent, very unscientific, but it’s a distraction from counting calories and quantities in general. Milk has water, fats, proteins, sugars, calcium, and more, Is it an ultraprocessed food, or is it exempt because a cow does the processing?
LikeLike
Good points. Natural components of whole foods do not count as ingredients even though whole foods are much more chemically complex than processed or ultra-processed ones. Humans process their foods through a series of steps know as digestion. The form of the mix of indigestible components of foods is not recognizable from the form entering our mouths.
LikeLike
Agreed. But re my own cow-human comment, cows don’t think but we do, and it’s the human decisions that we fear (humanipulation), not the ingredients. This explains the obsession with “natural,” despite the easy desanctification of the term by citing the “natural” Covid-19 virus, floods and hurricanes, snake venom and much more.
LikeLike
I agree, but the person selecting the food source for the cow are doing the same thing such as grass-fed or grain-fed.
LikeLike
Opens up another area. Cows and other food animals are humanipulated based on markets, geography and economics. Grass vs feedlots are results of these factors, and I doubt that the popular pro-grass image considers the price of meat. I doubt there is a significant nutritional difference, nor does Hall convince me that many ingredients increase my weight, but he does imply that those foods were more attractive or more convenient to eat.
LikeLike
Good question. Is milk exempt because the cow does the processing? The “food matrix” is a concept running through many of the commentaries I’ve read on NOVA. Milk from the cow (or goat or sheep) has many more known components than the ones you’ve referenced above. Just breaking down the fatty acids is extremely complex. And there may be other bioactive components that have not been identified and named. Milk has a food matrix that is greater than the sum of the known nutrient parts. And it is for that reason that milk is classified as a group 1 food.
LikeLike
The problem with that concept is that any food matrix starts breaking down when it hits the mouth and becomes a random mix of chemical components once it reaches the stomach and intestines. And how does the almost chemically pure sucrose which is heavily processed by Big Sugar maintain its group 1 status?
LikeLike