2019
Heart sound segmentation using bidirectional LSTMs with attention
IEEE Journal of Biomedical and Health Informatics
Six stages, four of them on the phone.
On the phone
1 · Capture
The app listens to the iPhone microphone exactly as it is. The processing phones normally apply to make voices clearer is switched off, because it would also erase the low, soft sounds a heart makes.
SourceThe iPhone microphone
ProcessingNone
On the phone
2 · Placement
Before the twenty seconds begin, a small model running on the phone answers one question: can I hear a heart? The field turns teal when the answer is yes, and the Record screen says what to change when it is no. Nothing is recorded or sent during this step.
On the phone
3 · Record
Twenty seconds, counted down on screen. Talking or moving restarts the count, so the recording that leaves the phone is a clean one. The recording is packed to a small file that keeps everything a heart sound contains and nothing else.
Duration20 s
SizeAbout 160 kB
In transit
4 · Send
The file travels over an encrypted connection to storage that belongs to your account, and the session is added to your list so it appears on any phone you sign into. If the connection drops, the recording stays on the phone until it is sent.
In the service
5 · Analyse
The analysis service finds every heartbeat in the recording as a sequence of S1, systole, S2 and diastole, and keeps only the beats it is confident about. For each accepted beat it measures the four durations and estimates blood pressure with a confidence. The summary the app shows is built from those beats.
Per beatS1, systole, S2, diastole
Also per beatBlood pressure, confidence
SummaryHeart rate, cycle, four durations, blood pressure
On the phone
6 · Compare and say
The app waits for the result, then compares it with your usual range. The range is yours: the band your own last four weeks of readings have drawn, once there are at least seven. The result becomes one sentence and one colour. Blood pressure that no beat could carry is shown as pending, never as a guess.
Usual range from7 readings in 28 days
VerdictsUsual, higher, lower
ColoursTeal, amber, slate
The four durations the service measures.
A phonocardiogram is the heart's mechanical timeline. Each accepted beat is cut at five instants and the gaps between them are the measurements. Move across the trace.
S1
Illustrative trace and timings, not a real recording.
| Measurement | What it is | Illustrative |
|---|---|---|
| S1 | The first heart sound: the mitral and tricuspid valves closing as the ventricles begin to contract. | 112 ms |
| Systole | From the end of S1 to the start of S2: the heart squeezing, blood leaving through the aortic and pulmonary valves. | 214 ms |
| S2 | The second heart sound: the aortic and pulmonary valves closing as the ventricles relax. | 96 ms |
| Diastole | From the end of S2 to the next S1: the heart filling. Longer at rest, shorter as the heart speeds up. | 478 ms |
| Cycle | One S1 to the next. Sixty thousand divided by the cycle is the heart rate. | 900 ms · 67 bpm |
In the app these appear on the Analysis screen for this recording, with a plus or minus for how much they varied across the accepted beats, and over time on the Sounds and Trends tabs.
Why sound, and not the electrical trace.
An electrocardiogram and a phonocardiogram answer different questions. Watches and patches now record the first very well. The second has stayed in the clinic, behind a stethoscope and a trained ear, which is the gap BeatAware is built for.
ECG
Electrical
When the heart's cells depolarise. Excellent for rhythm and rate. Silent about the valves and about the pressure the heart works against.
PCG
Mechanical
When the valves actually close and how long each phase lasts. Changes with filling, contraction and relaxation, and carries information an electrical trace does not.
Both
Timing
Each gives the length of the cycle, so heart rate agrees between them. Only the sound gives the four durations inside the cycle.
How the models learned to listen.
The approach behind the analysis is published. The segmentation work uses recurrent networks with attention to find S1 and S2 in noisy recordings; the interpretability work shows a classifier that explains which part of the sound it is reacting to; the domain-generalisation work is about models that keep working on a microphone they were not trained on.
2020
A robust interpretable deep learning classifier for heart anomaly detection without segmentation
IEEE Journal of Biomedical and Health Informatics
2020
Domain generalization in biosignal classification
IEEE Transactions on Biomedical Engineering
2022
Robust and interpretable temporal convolution network for event detection in lung sound recordings
IEEE Journal of Biomedical and Health Informatics
For clinics: the same engine, with a console and an API.
Healthcare teams can manage patients, upload recordings made with a phone or a basic electronic stethoscope, and download reports through a web console or a REST API. The analysis is the same one the app uses.
BeatAware is a wellness product, not a registered medical device. Its reports are for information and support a clinician's own assessment; they do not replace it.