Apple Watch Series 12 Heart Rate Study: Ranking All Wearables!
In a Nutshell
Apple's head-to-head study of 1,254 people found the Series 12 the most accurate optical heart-rate wearable, with the Pixel Watch 4 a close second and Huawei Watch 5 third across running, cycling, HIIT, strength work, and daily living; Garmin, Whoop, Galaxy Watch, and Oura Ring trailed. The ranking held whether Apple used mean absolute error or RMSE versus a Polar H10 ECG chest strap. Key caveats remain: dropout handling wasn’t disclosed, and the study was Apple-run, so independent verification is still needed.
These notes were generated by AI and may contain inaccuracies.
Apple claims that the new Apple Watch Series 12 and Ultra 4 have the most accurate heart rate sensing of any wearable from any brand. The study results show which competitors Apple considers best and worst, and the analysis examines what the results do and do not demonstrate.
The study enrolled 1460 participants with 1254 contributing data to the final analysis, representing just over 1200 people. Participants wore both an Apple Watch Series 12 on one wrist and a competing wearable on the other wrist. The Polar H10 ECG chest strap served as the reference device. The Apple Watch and competitor were randomly placed on left and right wrists, meaning the Apple Watch was approximately 50% of the time on the dominant hand and 50% on the non-dominant hand.
The testing setup is fundamentally similar to independent testing methods that compare wearables directly against an ECG chest strap, specifically the same Polar H10 model. Apple compared the Series 12 against the Garmin Forerunner 970, the Pixel Watch 4, the Huawei Watch 5, the Galaxy Watch 8, the Whoop Strap 5.0, and the Oura Ring 5.
Participants completed outdoor running, treadmill interval cycling, HIIT exercises, and strength training. These conditions were selected because they include many scenarios that challenge optical heart rate sensors, including cadence lock during running, gripping during cycling and weightlifting, and rapid heart rate changes during intervals.
Background heart rate measurements were tested during rest, desk work, meal prep, and walking. Apple primarily evaluated accuracy using mean absolute error and root mean square error (RMSE). Mean absolute error calculates how far each watch measurement is from the ECG reference, ignores whether measurements were too high or too low, and averages that error. RMSE is similar but gives more weight to larger errors. Results are essentially the same regardless of which metric is used, so RMSE is emphasized as it is the first metric shown.
Apple does not plot the absolute accuracy of the Apple Watch and competing devices. Instead, they subtract the two values from each other. Values left of zero indicate the Apple Watch performed better, while values right of zero indicate the competing device performed better.
For outdoor cycling, the average results across all participants showed the Apple Watch performing better than all devices except for the Pixel Watch 4, where there was no significant difference. In running tests, the second best device after the Apple Watch was the Pixel Watch 4, followed by the Huawei Watch 5, then the Garmin 970. For indoor running, the ranking was again Pixel Watch 4 as second best, followed by Huawei Watch 5, then the Whoop Strap.
The Pixel Watch and Huawei Watch consistently ranked as number two and three after the Apple Watch across all different exercises. This pattern is notable because these are smart watch companies rather than traditional sport companies. The results suggest that machine learning has become critical for accurate heart rate measurements, and older sports companies may need to catch up with companies that have invested heavily in machine learning and AI.
For strength training, the Apple Watch appears to perform significantly better than competitors. However, dropout rates are not addressed in the study results. It remains unclear whether the Apple Watch stops detecting during problematic moments or if there is some penalty for excluding measurements.
When all exercise results are summarized and ranked, the order is: Apple Watch first, Pixel Watch second, Huawei Watch 5 third, Garmin 970 fourth, Whoop Strap 5.0 fifth, Galaxy Watch sixth, and Oura Ring last. The Oura Ring's poor performance is attributed to the ring form factor being unsuitable for heart rate measurements.
For daily living measurements without starting specific exercises, the ranking remains consistent with Pixel Watch as second and Huawei Watch 5 as third. The Garmin 970 was very close in performance, while the Whoop Strap performed notably worse in this context.
In most cases, the Apple Watch performed best or equally well as competitors. However, during daily living measurements, there were several instances where the Pixel Watch actually outperformed the Apple Watch, and some cases where performance was essentially identical.
The study methodology appears sound and the calculations trustworthy. However, questions remain about dropout handling. Previous Apple Watch heart rate algorithms dropped out values when signal quality was poor, which differs from how other wearables typically handle measurements. This could potentially give the Apple Watch an advantage if bad quality signals are excluded while other devices continue reporting measurements.
Apple conducted their own study showing their device as best, which requires taking results with appropriate skepticism. However, Apple invested significant effort in both training their model and validating its performance. Notably, Apple did not avoid showing that the Pixel Watch performed quite well and consistently ranked as number two.
Bluetooth broadcasting was used for some devices to transmit signals, but it is unclear whether Bluetooth signals include timestamps with actual measurements or if timestamps are created at the moment of receipt. This timing difference could influence accuracy measurements and requires clarification from Apple.
Independent reviewers should verify these findings at smaller scale but with independent methodology. There is value in both manufacturer-conducted large-scale studies and independent smaller-scale verification.
Keep The Quantified Scientist in your library
Save the videos and channels worth coming back to, and find them again in one place.
