The Apple Watch Has a Problem
In a Nutshell
Apple's new Watch Series 12 and Ultra 4 introduce two controversial audio features that continuously listen to conversations: "Live Rewind" captures the last 15 seconds of audio via double-tap, and "Siri Recap" generates daily summaries of conversations for storage on your iPhone. The core issue is privacy—unlike stationary smart speakers, the watch enables non-consensual recording of conversations in public spaces without visible indicators or notifications to others. These features, powered by a new S11 chip with secure enclaves, appear designed to compete against emerging AI companion devices like those from OpenAI while leveraging Apple's massive installed base.
These notes were generated by AI and may contain inaccuracies.
The Apple Watch Series 12 and Apple Watch Ultra 4 received mostly minor updates overshadowed by iPhone Duo announcements and other tech hype. The updates specifically target two companies in different ways. The first target is obvious while the second is considered insane.
The market for screenless fitness trackers has more than doubled in the past year, largely due to Whoop holding more than half the market share. Competitors include Fitbit Air, Garmin Circa, and Amaze Fit. Apple chose not to create a dedicated screenless fitness band but instead updated the Apple Watch Series 12 to compete directly.
The Apple Watch Series 12 features a redesigned sensor array at the bottom with larger, more powerful, and more efficient LEDs and photodiodes. This enables more frequent sampling without degrading battery life. Background heart rate readings now occur every 5 seconds instead of every 5 minutes, matching competitive wearables. Apple published a paper claiming the watch offers the most accurate heart rate sensing among wearables, including rings.
The updated Health app uses AI to transform from a repository of mundane data into an organized display of overall health with actionable tips. This mirrors Whoop's approach for users seeking proactive health management beyond basic ring closures.
Whoop provides sleep scores and recovery scores to indicate rest levels and daily preparedness. Apple Health now offers equivalent sleep scores and readiness scores. Whoop includes a health span section showing "Whoop age" based on contributing factors, while Apple Health added a longevity section displaying "health age" with similar factor breakdowns.
Whoop offers an advanced lab section for comprehensive blood testing results upload. Apple Health introduced a metabolic health section with the same blood testing and upload functionality for detailed health insights.
Four audio intelligence features appear on Apple's website: music detection (existing), sound recognition (existing accessibility feature), live rewind (new), and Siri recap (new). The latter two features are considered the creepier additions.
Siri recap uses ambient listening to generate high-level notes from conversations throughout the day, saving them in the Siri app. Live rewind allows double-tapping the digital crown to access transcripts of the previous 15 seconds, which can also be saved to the Siri app.
The keynote demonstration showed live rewind capturing a waiter's lengthy daily specials list. The feature enables reading the list on the watch rather than requesting repetition.
The new features create an inescapable reputation that the Apple Watch constantly listens, representing a privacy nightmare. Apple states on their site that audio is not recorded or stored, cannot be accessed, individual speakers are not identified, and syncing uses end-to-end encryption.
The features require the new S11 chip with its secure exclave component. The microphone constantly captures raw audio, which gets written through the secure exclave that remains isolated from other S11 components. A machine learning algorithm in the secure exclave performs pattern recognition for sound signatures like doorbell chimes or songs.
Double-tapping the crown to trigger live rewind manually closes the secure exclave container and captures the last 15 seconds of audio. The audio encrypts and sends to the iPhone for decryption and transcription, then returns the transcription to the watch. Raw audio deletes from both devices afterward.
The microphone constantly captures audio, but the secure exclave prevents the S11 chip from accessing the data. While Apple claims this means no recording occurs, the constant audio capture and storage in the exclave could be considered recording despite the technical separation.
The microphone continuously receives audio, triggering capture when it detects conversation-like human speech patterns. The raw data encrypts, transcribes, and condenses into readable summaries saved and displayed on the iPhone at day's end. The feature is opt-in and can be gated to specific times like work hours.
The invasive aspect involves capturing conversations from non-consenting participants who have their discussions summarized and saved without their knowledge or permission.
The audio features target the physical AI companion market, including OpenAI's hardware development. OpenAI has recruited Apple employees and works with Jony Ive's design firm on a portable AI companion device expected to listen continuously and provide summaries and assistance.
Apple leverages its existing user base of hundreds of millions of iPhone users to maintain market position against potential new device categories that could replace smartphones.
The privacy concern differs from stationary devices like Google Home or Alexa because the watch accompanies users into public spaces, subjecting non-consenting individuals to surveillance. Unlike Meta glasses with visible recording indicators allowing people to remove themselves from situations, the Apple Watch operates quietly without indicators.
Apple addresses privacy through no speaker identification, no raw recording access by Apple, encryption, and audible chimes during live rewind activation. However, the Siri recap feature runs continuously without chimes or lights, providing no notification to nearby individuals when conversations are being summarized.
The watches launched with the S11 chip and secure exclave, but the audio intelligence features remain unreleased. When launched, they will debut in beta. The features were not included on new iPhones to avoid damaging the device's reputation. Rumors suggest future AirPods with cameras using similar privacy arguments about infrared emitters and encryption.
The question remains whether Apple can successfully explain and justify these features publicly, especially as competitors like OpenAI, Google, and Amazon may cross similar privacy boundaries.
Keep Marques Brownlee in your library
Save the videos and channels worth coming back to, and find them again in one place.
