Fitness trackers good for tracking heart rate but not calories: Stanford study

If you have an Apple Watch or a Fitbit Surge, their measurement of burned calories may be off.

Researchers from Stanford Medicine discovered six out of seven tested devices could not reasonably measure how many calories were being burned during exercise. The most accurate calorie counter was the Fitbit Surge, which was still off by a median error rate of 27 percent; the worst went to PulseOn, a Finnish fitness tracker, which was off by a whopping 93 percent.

In contrast to the devices’ inconsistency in counting calories, their heart rate measurement was accurate enough to be considered clinical quality. Six of seven devices — the exception was Samsung Gear 2 — recorded a median error rate of less than 5 percent. For both heart rate and calorie counting, the researchers were looking for an error rate under 10 percent.

“For the heart rate, we were pleasantly surprised. We are getting to the point where it’s getting valuable for physicians to follow heart rates on these devices,” said Anna Shcherbina, a co-author of the study. “For energy expenditure, we were surprised again but not in a positive way. We want the error rate below 10 percent, and none of the devices are quite there yet.”

Researchers measured both the heart rate and calorie counting for 29 male and 31 female participants while they were sitting, walking, running and cycling. To accurately measure burned calories in contrast to the devices, the researchers counted the participants’ carbon dioxide levels exhaled during the activities while factoring their age, gender and body mass index.

From August to November 2015, the researchers independently bought Apple Watch, Basis Peak, Fitbit Surge, Microsoft Band, Mio Alpha 2, PulseOn and the Samsung Gear S2 to make sure they could test the products fresh off the box.

It remains unclear exactly why counting calories was so inaccurate. Shcherbina wrote that gender, body mass index and even body hair and tattoos may affect the results. For example, male participants had a higher error rate than females. One hypothesis is that male body hair lessened the body contact between the devices and the person’s wrist and affected the energy expenditures.

As for why heart rates were measured more accurately, most devices now use a green LED light that shines into the skin to illuminate the blood flow and the shapes of blood vessels to get an image on how much the blood is pumping.

Of the seven companies involved in the test, three companies released lengthy comments to push back on the study. Apple, Microsoft and Intel, which owns the now-defunct Basis Peak, declined to comment. Samsung did not respond to a request for comment.

“Seeing so high errors in calories suggest that the authors may not have properly set all the user parameters,” said Markku Lankinen, PulseOn’s head of operations. “They do not report that they would have set the device settings before each subject to match the user. With PulseOn device, you would need to apply these user parameters in the application before exercising, and this seems not to have been done. This causes the EE estimates to be badly off.”

“Fitbit is committed to making the best clip and wrist-based activity trackers on the market and are confident in the performance of all of our trackers,” added a Fitbit spokesperson. “Overall, the success of Fitbit products comes from enabling people to see their overall health and fitness trends over time — it’s these trends that matter most in achieving their goals.”

For those looking for the most accurate calorie counter, co-author Dr. Mikael Mattson recommends retroactively counting calories after an exercise. For runners, for example, the cost for running is approximately 1 kcal/km per kg of body weight, regardless of speed.

But for those who want a more real-time counter, Shcherbina says fitness trackers are not a bad option. “You can get a good proxy,” said Shcherbina. “It’s reasonable to get a good estimate in energy expenditure but balancing that with having a non-invasive device.”

Photo: Apple Watch, along with six other devices, recorded inconsistent measurements in counting burned calories, according to a Stanford study. (Gary Reyes/Bay Area News Group)


Tags: , , , , ,


Share this Post

  • Makikiguy

    I measure the distance in feet on Google Maps and then divide by the length of my stride.