How Cry Detection Works in a Baby Monitor App

A basic sound-activated monitor only asks one question: is it loud? That's why a slammed door or a passing truck sets it off. Telling crying apart from everything else takes a few more questions. Here's how Babyphone answers them, and where the approach falls short.

Step 1: What counts as loud in this room?

A fixed volume threshold doesn't work. A quiet nursery and a room with a humidifier have very different baselines.

So the detector continually estimates the room's noise floor. It drops quickly when the room gets quieter and rises slowly when the room gets louder. If you switch on a fan, it fades into the background after a few minutes. Sounds that look like crying never raise the floor, so a long crying spell can't teach the detector that crying is normal.

A sound counts as loud when it's sufficiently above this floor. The sensitivity setting controls how far above.

Step 2: Is the sound pitched like a baby?

Crying is a voiced sound, so it has a clear pitch, known as its fundamental frequency. Babies' vocal tracts are small, which puts that pitch well above an adult's.

Research on infant cries generally puts healthy cries at around 200 to 600 Hz, with typical averages near 400 to 500 Hz. Adult speech is usually much lower.

For each short slice of audio, the detector compares the signal with delayed copies of itself. When the signal lines up with a copy delayed by a certain amount, that delay reveals the pitch. If the slice is clearly periodic and the pitch is between 250 and 800 Hz, the slice might be crying.

Step 3: Does it have harmonics?

A smoke alarm or a beeping microwave can be in the same pitch range, but it's close to a pure tone. A human voice contains energy at multiples of its base pitch, called harmonics.

The detector measures the energy at twice and three times the pitch. If those harmonics are too weak, the sound isn't treated as crying.

Step 4: Does it behave like crying over time?

One slice proves nothing. The detector looks at the last four seconds and asks two questions:

• Do crying-like slices make up at least 45% of that window? Speech has many gaps and consonants, while crying is long and sustained.
• Does the pitch move? Crying has a melody of rises and falls. A steady electronic tone doesn't.

If both are true, the state changes to crying and stays there for a few seconds, so that a breath between wails doesn't flip it back and forth. On our test signals, this takes about two seconds.

Why it runs on the phone

All of this runs on the device near your baby, in the browser. The analysis doesn't need to send audio anywhere, and the only thing that reaches the parent's phone as a result is a small status: quiet, noise or crying.

Where it gets things wrong

We tested the detector with synthetic signals modeled on crying, speech, white noise, tones and door slams. We haven't validated it against a clinical recording set, and this approach has known blind spots:

• Older siblings and high-pitched voices can fall into the same pitch range, especially when sustained.
• Singing, some music and TV can be periodic and melodic enough to trigger it.
• Pets can sound surprisingly similar to crying.
• Soft fussing may be classed as noise rather than crying, or not noticed at all.
• A microphone far from the crib weakens the harmonics that the detector depends on.

That's why you can choose to be alerted for any noise, keep continuous sound on, and adjust the sensitivity. Think of cry detection as a useful nudge. It isn't a guarantee, and it doesn't replace good placement or an adult who's within earshot.