We Studied Sleep With 15 Wearables. This is the WINNER!

Study Overview and Initial Results00:00

Together with the University of Salzburg, ran one of the biggest wearable sleep studies ever to determine which wearable best tracks sleep stages: deep sleep, light sleep, REM sleep, and awake time. Initial analysis shows clear winners and losers. Overview plot positions best devices top right: horizontal axis Cohen's Kappa, vertical axis average sensitivity (agreement with individual sleep stages). Clear winners: Apple Watch, Oura Ring, latest Whoop algorithm. Below: old Fitbit algorithm, old Whoop. Worst: Garmin, Circular Plus Ring. Included Sleep 2 (University of Salzburg app), take with grain of salt due to vested interest. Check devices (all light sleep average night, median polysomnography) perform worse than wearables.

Top Performers and Metrics Explanation01:02

Devices: Oura Ring, Apple Watch, Fitbit, Whoop, Garmin. Rob is post-doctoral scientist specializing in biological data analysis. Sleep 2 did best by these metrics, but potential bias. Manufacturers likely optimize for Cohen's Kappa and average sensitivity. Apple, Oura, Whoop (new) outperform Garmin. Fitbit has newer algorithm (untested here) that shows improvement in early testing, potentially top performer.

Study Setup02:31

20 participants (18 after 2 dropouts), tracked 5 days: Monday-Friday. Each night different focus:

• Night 1: Habituation to polysomnography (PSG) reference device at home.
• Night 2: Normal sleep.
• Night 3 (Wednesday): Sleep restriction, awake ~3:00 a.m. for two 10-minute slots (total 10-20 minutes).
• Night 4 (Thursday): Normal recovery sleep (higher total sleep time).
• Night 5 (Friday): Extended sleep. Wore duplicate devices: 3 left arm, 3 right arm (two Garmin, two Fitbit, two Oura Rings). Whoop on biceps with Polar Verity Sense and Polar H10 (for Sleep 2). All nights with PSG.

Sleep Stage Percentage Accuracy06:31

Plot shows device prediction minus PSG reference for % of each stage (two devices per type, left/right arm):

• Awake: All predict too little except Oura Ring (roughly accurate), Whoop especially low.
• Light sleep: Whoop, Sleep 2, Oura roughly accurate; others too much.
• Deep sleep: Sleep 2, Oura, Garmin, Fitbit, Circular accurate; Apple Watch too little (updated algorithm may improve).
• REM sleep: Whoop, Sleep 2, Oura, Fitbit, Circular accurate; Garmin too little, Apple Watch too much. Best for percentages: Oura, Sleep 2. All predict too little awake except those two. Apple low deep, Garmin low REM.

Day-to-Day Variation and Correlations09:01

Key: Does device detect relative changes (more/less deep sleep than your average)? Normalized per person over 5 nights (individual night minus personal average). Example: Oura agrees with PSG on deep/REM deviations, some noise. Correlation plot (agreement on relative amounts):

• Sleep 2 best (lab bias possible).
• Oura: Excellent REM/light, decent deep, low awake.
• Whoop (new): Excellent REM/light, okay deep, poor awake.
• Apple Watch: Good awake/light/REM, poor deep.
• Garmin, Fitbit worst overall (Fitbit good REM, Garmin okay light). Garmin/Fitbit one level below others. No super bad devices included.

Individual Variation13:00

Plot: average sensitivity (x) vs. worst sleep stage sensitivity (y), dots per individual.

• Sleep 2 (orange), Oura (pink), Whoop old/new (blue), Apple (inside blue), Garmin (red), Fitbit (green), Circular (black). High individual variation: Oura big spread, Whoop/Apple low spread. Garmin/Fitbit poor, high spread.

Per-Person Rankings14:01

Ranked devices 1-6 per person, counted frequencies:

• Sleep 2 always #1 (lab bias).
• Oura often #2/3.
• Apple bimodal: some #2/3, others #5/6.
• Whoop stable ~#4.
• Fitbit ~#4/5.
• Garmin worst of popular ones.
• Circular Plus worst overall.

Overall Performance Plots16:01

Average sensitivity (x, equal weight 4 stages) vs. worst stage sensitivity (y), healthy individuals only (little difference). Oura stands out: consistent >65% worst stage, no bad stage. Order: Oura > Whoop new > Apple > Fitbit old > Garmin ~ Circular. Best: Apple, Whoop, Oura, probably updated Fitbit. Garmin not terrible.

Performance on Bad Sleep Night18:00

Night 3 (interrupted/restricted): Similar order, Fitbit worse than Garmin vs. normal night. Oura, Apple, Whoop new still top. Performance stable on bad nights (mild disruption). Untested: severe sleep disorders.

Final Notes19:30

Initial analysis (preprint by study lead Pavlo available). Full video/reanalysis coming. Thanks to University of Salzburg (Pavlo, Lauren). Affiliate links for Whoop, Oura, Eight Sleep Pod, Amazon.