vibe code the initial impl
This commit is contained in:
@@ -1,32 +1,149 @@
|
||||
# ESP32-S3 INMP441 noise indicator
|
||||
# Door buzzer detector (ESP32-S3, ESP-IDF, spectral template matching)
|
||||
|
||||
This ESP-IDF sample reads audio from an INMP441 I2S microphone and indicates
|
||||
the smoothed sound level (RMS) using the ESP32-S3 board's onboard RGB LED. The
|
||||
microphone's left channel is used because its L/R pin is grounded.
|
||||
Listens through an I2S MEMS mic, compares the *shape* of the spectrum of every 32 ms frame to a stored
|
||||
buzzer template (cosine similarity of a 32-band log-spectrum fingerprint), and sends an ntfy.sh push
|
||||
notification when a sliding window says "that was the buzzer". No machine learning.
|
||||
|
||||
The LED fades from off to green as the smoothed RMS rises from 0 to 100,000.
|
||||
From 100,000 to 1,000,000 RMS, it smoothly changes from green to red; at or
|
||||
above 1,000,000 it remains red. These thresholds are 10 times higher than the
|
||||
previous settings to make red less likely during ordinary background noise.
|
||||
Written in C (C11), as your file list (`main.c`, ...) implied. ESP-IDF API usage follows v5.x/v6.0 conventions
|
||||
(`driver/i2s_std.h`, `esp_driver_*` components, esp_console REPL).
|
||||
|
||||
The thresholds are defined by `GREEN_NOISE_RMS` and `RED_NOISE_RMS` near the
|
||||
top of [main/main.c](main/main.c). Adjust them after checking the RMS values in
|
||||
the serial monitor for your microphone and environment.
|
||||
## Project tree
|
||||
|
||||
## Wiring
|
||||
```
|
||||
door_buzzer/
|
||||
├── CMakeLists.txt
|
||||
├── sdkconfig.defaults
|
||||
├── README.md
|
||||
├── tools/
|
||||
│ ├── make_template.py # recording -> template.h + test_vector.h
|
||||
│ └── host_test/ # optional PC test of fingerprint.c + detector.c
|
||||
└── main/
|
||||
├── CMakeLists.txt idf_component.yml Kconfig.projbuild
|
||||
├── feature_params.h # single source of truth (parsed by Python)
|
||||
├── template.h test_vector.h # placeholders until make_template.py runs
|
||||
├── board_config.h # pin #defines (from Kconfig)
|
||||
├── main.c # init, BOOT button
|
||||
├── audio_in.c/h # I2S std RX, frame pool, capture task (core 1)
|
||||
├── fingerprint.c/h # window, band edges, FFT, fingerprint
|
||||
├── matcher.c/h # active template (NVS > built-in > none), cosine score
|
||||
├── detector.c/h # pure IDLE/TRIGGERED/COOLDOWN state machine
|
||||
├── calibration.c/h # learn/adapt/selfcheck/test/background/autothresh
|
||||
├── config_store.c/h # runtime parameters in NVS
|
||||
├── processing.c/h # features -> score -> detector task, stats, monitor
|
||||
├── console.c/h # serial commands
|
||||
├── notifier.c/h # Wi-Fi w/ backoff + ntfy HTTPS task
|
||||
└── led.c/h # WS2812 status LED
|
||||
```
|
||||
|
||||
| Module | Pin | ESP32-S3 |
|
||||
| --- | --- | --- |
|
||||
| INMP441 | SCK / BCLK | GPIO 3 |
|
||||
| INMP441 | WS / LRCLK | GPIO 4 |
|
||||
| INMP441 | SD | GPIO 5 |
|
||||
| INMP441 | L/R | GND |
|
||||
| INMP441 | VDD | 3.3V |
|
||||
| INMP441 | GND | GND |
|
||||
| Onboard RGB LED | Data | GPIO 48 (on supported board variants) |
|
||||
Your staged build order maps onto this as: (1) `feature_params.h`, `audio_in`, (2) `fingerprint`, `selftest`,
|
||||
`make_template.py`, (3) `matcher`, `calibration`, `config_store`, (4) `detector`, `processing`, (5) `notifier`,
|
||||
(6) watchdog / reconnect / LED in the same files. The final tree is what is delivered; each layer only depends on the ones before it.
|
||||
|
||||
## Build and run
|
||||
## Wiring (INMP441)
|
||||
|
||||
Open this folder as an ESP-IDF project, select the `esp32s3` target, then build,
|
||||
flash, and open the serial monitor. The firmware uses ESP-IDF's I2S driver and
|
||||
the `espressif/led_strip` component.
|
||||
| INMP441 | ESP32-S3 | Note |
|
||||
|---|---|---|
|
||||
| VDD | 3V3 | |
|
||||
| GND | GND | |
|
||||
| SCK | GPIO3 | BCLK (`BUZZ_I2S_BCLK_GPIO`) |
|
||||
| WS | GPIO4 | (`BUZZ_I2S_WS_GPIO`) |
|
||||
| SD | GPIO5 | data out of mic (`BUZZ_I2S_DATA_GPIO`) |
|
||||
| L/R | GND | left slot (to VDD -> change slot to Right in menuconfig) |
|
||||
|
||||
On-board: RGB LED on GPIO48 (WS2812), BOOT button GPIO0. LED: red = needs template, dim blue = listening,
|
||||
amber = calibration session, magenta = alert/cooldown.
|
||||
|
||||
## menuconfig
|
||||
|
||||
```
|
||||
idf.py set-target esp32s3
|
||||
idf.py menuconfig # "Door Buzzer Detector"
|
||||
```
|
||||
* **Wi-Fi**: SSID + password.
|
||||
* **ntfy**: server (default `https://ntfy.sh`), **topic** (pick something unguessable: anyone who knows it can read it),
|
||||
optional heartbeat topic and interval (hours, 0 = off). Subscribe to the topic in the ntfy phone app.
|
||||
* **Hardware**: pins, I2S slot, `Right shift` (default 14 = 4x digital gain for the quiet INMP441).
|
||||
* **Detector defaults**: threshold, loudness gate, N/M, cooldown (default 5 ms). The existing N-of-M sliding window smooths frame matches; because frames arrive every 32 ms, cooldown/re-arm timing is effectively frame-granular. All can also be changed live with `set` and persist in NVS.
|
||||
* Console: defaults to the **UART** USB port. For the native USB port set *Component config > ESP System Settings > Channel for console output*
|
||||
to *USB Serial/JTAG*.
|
||||
|
||||
## Build / flash
|
||||
|
||||
```
|
||||
idf.py set-target esp32s3
|
||||
idf.py build flash monitor
|
||||
```
|
||||
|
||||
## Workflow
|
||||
|
||||
**a. Make the template from your recording**
|
||||
```
|
||||
pip install numpy scipy
|
||||
python tools/make_template.py --buzz buzzer.wav --background quiet_room.wav tv_talking.wav
|
||||
# optional: --segments 1.2-2.0,5.5-6.4 (otherwise auto-detected: frames > 4x noise floor)
|
||||
```
|
||||
It prints the selected time ranges, buzzer score distribution, a text bar chart, background score stats, a recommended threshold,
|
||||
and an estimated false-trigger rate per hour, then writes `main/template.h` and `main/test_vector.h`. It refuses with < 20 buzzer frames and warns loudly if background and buzzer scores overlap.
|
||||
|
||||
**b. Build, flash, verify**
|
||||
```
|
||||
buzz> selftest # must print PASS (C == Python, tolerance 1e-3)
|
||||
buzz> selfcheck # press the real buzzer for 10 s; verdict good / weak / poor
|
||||
```
|
||||
**c. If the match is weak/poor** (your recording used a different mic):
|
||||
```
|
||||
buzz> adapt # 60 s: blends live buzzer 50/50 into the PC template (adapt 0.7 = weight of NEW sound)
|
||||
or buzz> learn # 60 s: replace the template completely (also: short-press BOOT)
|
||||
buzz> background # 60 s of normal ambient sound, do NOT press the buzzer
|
||||
buzz> test # 10 s, press the buzzer
|
||||
buzz> autothresh # threshold = midpoint of background p99 and buzzer p5, saved
|
||||
```
|
||||
**d. Live test**
|
||||
```
|
||||
buzz> monitor on # timestamp, RMS, score, MATCH flag, state
|
||||
buzz> alert # test notification
|
||||
buzz> status # template source, timing (must be << 32 ms), Wi-Fi, notification counters
|
||||
```
|
||||
`learn reset` clears the NVS template and falls back to `template.h`. During any calibration session the detector is
|
||||
paused (pressing the buzzer will not send alerts).
|
||||
|
||||
### Console reference
|
||||
`selftest` `selfcheck` `learn [reset]` `adapt [w]` `background` `test` `autothresh` `monitor on|off`
|
||||
`set <param> <value>` `get [param]` `status` `fingerprint` `alert` `reboot`.
|
||||
Parameters: `threshold min_rms win_n win_m cooldown_ms noise_mult adapt_weight monitor_ms heartbeat_h`.
|
||||
|
||||
## Design notes
|
||||
|
||||
* **Template priority at boot:** NVS-learned > compiled-in `template.h` > "needs template" (instructions printed, LED red). `status` shows which.
|
||||
* **Fingerprint:** DC removal per frame -> RMS -> samples/32768 x periodic Hann -> 1024-pt FFT (esp-dsp) -> *mean* power per band over
|
||||
32 log-spaced bands -> ln(P+1e-10) -> subtract mean -> (FP_DETREND) subtract least-squares line over band index -> L2 normalise.
|
||||
Score = dot product (cosine).
|
||||
* **Parity:** `feature_params.h` is parsed by Python; band edges use double math and `floor(x+0.5)` on both sides. I verified on a PC
|
||||
(real `fingerprint.c` + stub FFT vs numpy) a max fingerprint error of ~1e-6, also with other NUM_BANDS / FP_DETREND values.
|
||||
* **Concurrency:** audio task (core 1, prio 10) -> pool of 8 frame slots + two index queues (no allocation; drops and logs if processing lags)
|
||||
-> processing task (core 1, prio 8). Network and console tasks are separate; the notifier has its own queue. Mode and session data
|
||||
are guarded by a mutex; template and config by mutexes. Audio and processing tasks are on the task watchdog (5 s, panic -> reboot).
|
||||
* **Notifier:** exponential Wi-Fi reconnect (1 s .. 60 s); up to 4 delivery attempts with backoff, then dropped and logged.
|
||||
* Simplifications chosen on purpose: one `TEST_CAL` mode serves `test` and `selfcheck`; calibration results (background/buzzer stats) live in RAM only
|
||||
(re-run after reboot before `autothresh`); `TEMPLATE_SUGGESTED_THRESHOLD` is shown in `status`, not applied automatically.
|
||||
|
||||
## Assumptions
|
||||
|
||||
* INMP441-style mic: 24-bit in a 32-bit Philips slot, mono. 16 kHz, hop 32 ms.
|
||||
* `min_rms` is in int16 units *after* the shift/gain, so it depends on `BUZZ_AUDIO_SHIFT`. Default 200 is a guess; check `monitor on` while the buzzer sounds vs. silence and adjust.
|
||||
* `learn`/`adapt` gate = max(median RMS of the first second x `noise_mult`, `min_rms`) so stay quiet in the first second. `test`/`selfcheck` count frames with RMS >= `min_rms`.
|
||||
* Verdict: **good** if the 5th-percentile score >= threshold, **weak** if only the median passes, **poor** otherwise.
|
||||
* GPIO3 is an ESP32-S3 strapping pin (JTAG source); it works as an I2S clock after boot but avoid a heavy pull-up/down on the mic line.
|
||||
|
||||
## Tuning tips
|
||||
|
||||
* **False alarms:** raise `threshold` (run `background` + `autothresh`), raise `min_rms`, or raise `win_m` (e.g. 12/15). Door slams and speech are loud
|
||||
but rarely hold a buzzer-shaped spectrum for 0.3+ s.
|
||||
* **Missed buzzes:** `selfcheck` first. Weak/poor -> `adapt`/`learn`. Then lower `threshold` slightly, lower `win_m` (8/15), lower `min_rms`
|
||||
(the buzzer may be quieter at the mic than you think; use `monitor on`).
|
||||
* **Mic mismatch:** the PC recording mic and the INMP441 have different frequency responses. Detrending cancels a tilt but not bumps; `adapt` fixes
|
||||
residual differences without throwing away a good template.
|
||||
* **FP_DETREND:** keep on by default. Turn it **off** (`#define FP_DETREND 0`) if the buzzer's own spectral tilt is its most distinctive feature
|
||||
(e.g. a very low, dull buzz vs. bright background noise) and `background`/`test` overlap. Then re-run `make_template.py` and rebuild: it changes the fingerprint, so an old template/NVS template is invalid until relearned (`learn reset` + `learn`).
|
||||
Changing any `feature_params.h` value invalidates stored templates the same way; NVS templates with a different band count are ignored automatically.
|
||||
* Typical processing time is far below the 32 ms budget; `status` prints the measured average and maximum.
|
||||
|
||||
Reference in New Issue
Block a user