I recently came across yet another example of why observational studies tend to suck. You’ve probably seen the headlines, like this one from USA today: Men Who Can Do More Than 40 Push-Ups Far Less Likely To Develop Heart Disease.
Let’s take a look at the article:
Here’s one way to predict your heart health: get down and give me 41. A new study finds that men who can perform at least 40 push-ups in one attempt are much less likely to suffer from heart disease within the next 10 years.
Researchers from the Harvard T.H. Chan School of Public health say their report is the first to show how push-up capacity is linked to heart disease. They found that middle-aged men who can log more than 40 push-ups in a single try have a 96% reduced risk of developing the potentially deadly condition and other related ailments, such as heart failure, compared to those who can complete no more than 10 push-ups.
A 96% reduced risk?! Okay, guys, get on the floor and start doing push-ups! Having strong pecs and triceps obviously prevents heart disease!
I was of course suspicious when I saw this study came from the Harvard School of Public Health, which we ought to rename Meaningless Observational Studies R Us. Sure, it’s a good idea to stay in shape, and exercise no doubt protects against heart disease to some extent. But a 96% reduction based on the ability to do more push-ups? Something doesn’t smell right.
In a speech I gave many years ago, I highlighted the weakness of observational studies by pointing out that men who are bald are much more likely to suffer a heart attack than men sporting a full head of hair. If we applied Harvard School of Public Health logic, we would assume baldness somehow causes heart disease. Eventually, we’d end up with products like this:
But of course, the reason bald men have more heart attacks is that men lose their hair as they get older. They’re also more likely to suffer a heart attack as they get older. So baldness is “linked” to heart disease. I suspected the push-up study was based on similar nonsense. If we dig into the data, we might find the “link” exists because younger men can do more push-ups. I’m sorry to say I was right.
Here’s more about the study from USA Today:
For their study, the authors reviewed health data from 1,104 active male firefighters taken annually from 2000 to 2010. At the start of the study, the average participant was about 40 years old with an average body mass index of 28.7. The firefighters were tasked with performing as many push-ups as they could, and their treadmill tolerance was also tested.
By the end of the study period, 37 participants suffered from a heart disease-related condition — and 36 of those men weren’t able to log more than 40 push-ups in the initial test.
The average participant was 40 years old. Uh-huh … now let’s look at a table from the study giving us more detail on those participants:
Well, how about that? The average participant may have been 40 years old, but the mean age of the firefighters who could do more than 40 push-ups was 35. Among those who could do 10 or fewer push-ups, the mean age was 48. Big shock. Men lose their ability to do push-ups as they age. If we start with a group of 35-year-olds and another group of 48-year-olds, which group is going to suffer more heart attacks during the next 10 years?
But that’s only part of what makes this a meaningless study. One of the greatest risk factors for suffering a heart attack is smoking. Take a look at the figures I highlighted at the bottom of the chart. Among the men who could do more than 40 push-ups, just 6.7% were current smokers when the study began. Among those who could do 10 or fewer push-ups, 24% were current smokers. Or to use a related bit of data, among the more than 40 push-ups group, 69% were non-smokers when the study began. Among the 10 or fewer group, only 45.3% were non-smokers.
In the study itself, the authors of course state they applied regression models to account for age and BMI, blah-blah-blah. Interestingly, they don’t mention applying a regression model to account for smoking. Or perhaps they did, but chose not to mention the results.
In any case, just balancing for age and BMI presents a different picture:
Even after adjusting for age and BMI, we observed an independent association of push-up capacity with CVD outcomes. Increased capacity was associated with a lower risk for CVD outcomes, with the comparison of the 21- to 30-push-ups group vs the 0- to 10-push-up group being statistically significant (hazard ratio, 0.25; 95% CI 0.08-0.76), although the other group comparisons did not reach statistical significance.
Adjust for age and BMI (again, no mention of adjusting for smoking), and the only significant difference in push-ups vs. heart disease is between the 10 push-ups or fewer group and the 21 to 30 push-ups group. So much for that 96% reduction in heart attacks for men who can do more than 40 push-ups vs. men who can’t do more than 10.
This study doesn’t actually tell us diddly about the ability to do push-ups versus the likelihood of developing heart disease. It simply tells us that younger men and men who don’t smoke are less likely to develop heart disease during the next 10 years. And by the way, they can also do more push-ups. Duh. And yet in the media articles, we get quotes like this:
“Our findings provide evidence that push-up capacity could be an easy, no-cost method to help assess cardiovascular disease risk in almost any setting,” says the study’s first author, Justin Yang, an occupational medicine resident at the school.
No, your findings provide evidence that observational studies from Harvard are usually meaningless garbage dressed up as science. It’s a wonder anyone keeps funding these turkeys.
