Loading files
This file is pinned to the selected release. Download the original to open it in your design tool.
# Patternflow Firmware
Arduino-based firmware for the ESP32-S3 powering Patternflow. One image serves every board generation — the pin map is identical on v2.x, v3.0 and v3.9.
The firmware handles the ESP32-S3 DMA driver for the HUB75 LED matrix, reads four rotary encoders to control generative patterns, and supports Arduino OTA for wireless updates.
> ⚠️ **Panel compatibility.** This firmware drives the panel directly from the ESP32-S3, so the panel's **driver IC** decides whether it lights up: classic shift-register parts work, S-PWM "video wall" parts stay completely dark, and "HUB75E" on a listing proves nothing. The list of chips, and how to check before buying, is **[docs/panel-compatibility.md](../docs/panel-compatibility.md)**; the driver is selected below via `PANEL_PROFILE`.
## Reading order
This file is long because it is the firmware's reference. Most people need one part of it:
| you want to | read |
| --- | --- |
| build and flash the firmware locally | [Setup](#setup) — the board package, PlatformIO, the libraries |
| write a feature or cut an edition | [`docs/EDITIONS.md`](../docs/EDITIONS.md) first, then [Adding features — the constraints that matter](#adding-features--the-constraints-that-matter) here |
| write a pattern | [`PATTERN_GUIDE.md`](../PATTERN_GUIDE.md) (nothing here is needed); the hands-on route is [`CUSTOM_PATTERNS.md`](CUSTOM_PATTERNS.md) |
| know what a pattern can call | [Foundation modules](#foundation-modules) |
| know how `.pfm` modules load and what they cost | [Loadable pattern modules](#loadable-pattern-modules-pfm) |
| tune brightness, calibration, refresh | [Configuration](#configuration-configh) |
| know what the knobs and buttons do | [Controls](#controls), [Sleep mode](#sleep-mode) |
| drive the panel from other software | the contracts in [`docs/`](../docs/README.md): HTTP, OSC, MIDI, MQTT, audio |
| edit a console page | [`patternflow/console/README.md`](patternflow/console/README.md) |
## Building your own firmware
One image ships on the board, and it is not the only one you can run. Features
are modules in this tree; a build is a choice of which ones compile in; a
published build is an **edition**, installed from
[the shelf](https://patternflow.work/editions) in one click, with your
patterns and settings intact.
**[docs/EDITIONS.md](../docs/EDITIONS.md)** is where that starts: what a
feature is, how to write one, how to cut an edition, and the promises every
build keeps so switching stays safe. Read it before writing a feature.
## Setup
### Required board package
- ESP32 by Espressif Systems — **2.0.x, not the latest 3.x** (see below)
> ⚠️ **The core version decides how much RAM your patterns get.** Core 3.x
> ships ESP-IDF 5.5, which occupies about **71 KB more internal RAM before the
> sketch even starts** than core 2.x / IDF 4.4 — the gap is already there at the
> first heap reading in `setup()`, before Wi-Fi is up, and no sdkconfig setting
> reverses it (the Wi-Fi buffer flags are identical between the two prebuilts).
> Measured on the same board with the same source, free internal heap once the
> network services are up:
>
> | | free | largest block |
> |---|---|---|
> | core 3.3.8 (IDF 5.5) | 15,320 B | 7,668 B |
> | core 2.0.17 (IDF 4.4) | **98,708 B** | **90,100 B** |
>
> That largest-block number is the hard ceiling on a loadable module: a `.pfm`'s
> `.text` has to land in one contiguous internal executable block, because the
> S3 cannot execute loaded code from PSRAM. On core 3.x a 12 KB pattern is
> simply refused; on core 2.x it loads and runs. See
> [Internal RAM is the budget](#internal-ram-is-the-budget-everything-else-is-roomy).
### Build with PlatformIO (recommended)
`platformio.ini` in `patternflow/` pins core 2.x and is what the measurements
above come from. It also builds in ~90 s against the IDE's several minutes.
```bash
cd firmware/patternflow
pio run # → .pio/build/firmware/firmware.bin
pio run -t upload # or flash it yourself at 0x10000
```
`toolchain/sync_ino_to_src.py` copies the sketch into `pio_src/` and clones the
libraries below into `lib/` on first run, so there is nothing to install by
hand. On Windows, build from a path with no non-ASCII characters, or point
`PLATFORMIO_BUILD_DIR` at one — the xtensa linker cannot write outputs
underneath a path it cannot encode.
The Arduino IDE path still works and is documented below; it is what the older
guides describe. Just install core **2.0.x** rather than the latest.
### Arduino IDE board settings
- **Board:** ESP32S3 Dev Module
- **PSRAM:** OPI PSRAM
- **Flash Size:** 16MB
- **Partition Scheme:** 16M Flash with PSRAM-aware partition
- **USB CDC On Boot:** Enabled
- **Upload Mode:** UART0 / Hardware CDC
> **Why CDC On Boot matters.** It decides whether `Serial` is the native USB
> peripheral or UART0 — and therefore which of the two USB-C ports answers the
> browser flasher's Wi-Fi setup (Improv speaks over `Serial`). Enabled puts it
> on the **left/native `USB`** port, which is what the released firmware does
> and what §8.1 of the build guide tells people to use. Building with it
> disabled produces firmware that flashes fine but whose Wi-Fi setup only
> appears on the other socket.
### Required libraries
PlatformIO clones these for you (`lib_deps` in `patternflow/platformio.ini`
is the authoritative list). For the Arduino IDE, install them via the
Library Manager:
- `ESP32-HUB75-MatrixPanel-DMA` (for driving the matrix)
- `Adafruit GFX Library` (dependency)
- `WebSockets` by Markus Sattler / Links2004 (audio-react, Audio edition)
- `NimBLE-Arduino` (the `ble` feature — in the tree, in no edition)
- `AppleMIDI` by lathoub (network MIDI, Audio edition)
> The Arduino IDE builds the default firmware. It does not build the editions:
> their feature set is chosen by `firmware/bundles/<name>/features_local.h`,
> which only `bundles/build.sh` wires in. Use PlatformIO for anything that is
> not the plain device.
> The HUB75 library is also **vendored** under `patternflow/src/hub75/`, and that
> copy is the one that compiles — it carries Patternflow's additions
> (`resumeDMAoutput()` for sleep, the brightness hook the power clamp uses). The
> installed/cloned copy only needs to satisfy headers, so its exact version does
> not matter. See `src/hub75/VENDORED.md`.
OSC uses the ESP32 Arduino core's built-in `WiFi` and `WiFiUdp` libraries, so it does not require an extra OSC library. The MQTT client (`pubsubclient`) and the HUB75 driver are vendored under `patternflow/src/`.
## Project layout
```
firmware/
├── patternflow/ # The sketch (PlatformIO project; open patternflow.ino)
│ ├── patternflow.ino # Main sketch: input routing, mode dispatch — THE CORE
│ ├── config.h # Hardware configuration (pin mappings, limits, LED calibration)
│ ├── net_config.h # Wi-Fi / Improv / OTA / self-update defaults (core only)
│ ├── pattern_registry.h # Presets compiled in (Origin) + .pfm modules discovered on FATFS
│ ├── platformio.ini # The build. bundles/build.sh drives it.
│ ├── presets/ # Curated presets (35). Only Origin is compiled in; the
│ │ # rest ship as the Basics pack (web/public/packs/)
│ ├── abi/ # Host ⇄ module contract for loadable patterns
│ │ ├── pf_abi.h # Frozen C ABI: PFHostAPI, PFPatternModule, PF_ABI_VERSION
│ │ ├── pf_params.h # The absolute parameter bus
│ │ └── pf_module.h # Module SDK — the ONE header a .pfm pattern includes
│ ├── patternflow_secrets.example.h # Template (copy to patternflow_secrets.h)
│ ├── src/ # Foundation — the core, feature-free (see docs/EDITIONS.md)
│ │ ├── core_display.h # HUB75 driver init + refresh-rate config
│ │ ├── core_encoders.h # Encoder ISRs + InputFrame contract
│ │ ├── core_canvas.h # 128×64 RGB888 framebuffer + gamma/WB/sat
│ │ ├── core_math.h # PFMath — shared verbatim with modules
│ │ ├── core_color.h # PFColor — shared verbatim with modules
│ │ ├── core_noise.h # PFNoise — shared verbatim with modules
│ │ ├── core_mem.h # PSRAM-first allocator (PFMem)
│ │ ├── core_tables.h # Panel-space polar tables (host-owned, 32 KB each)
│ │ ├── core_module_loader.h # ELF loader/relocator for .pfm modules
│ │ ├── core_power.h # Total power clamp (measures demand, caps brightness)
│ │ ├── core_sleep.h # Sleep mode: panel off, board idle, still on the network
│ │ ├── core_wifi.h # Multi-network Wi-Fi (up to 5 saved, tried in order)
│ │ ├── core_improv.h # Improv-Serial provisioning from the browser flasher
│ │ ├── core_http.h # The one port-80 WebServer every page registers on
│ │ ├── core_net_task.h # The network task on Core 0: Wi-Fi, HTTP, Improv, OTA
│ │ ├── core_loop_sync.h # Hands a handler's frame-touching work to the loop task
│ │ ├── core_bus.h # Absolute knob bus (0..1000 wire scale)
│ │ ├── core_web_update.h # Browser self-update (/update)
│ │ ├── core_patterns_http.h # Module manager (/patterns + /api/patterns)
│ │ ├── core_status_http.h # Diagnostics (/status + /api/status)
│ │ ├── core_wifi_http.h # Wi-Fi manager (/wifi + /api/wifi)
│ │ ├── *_index.h # PROGMEM HTML for the pages above (generated from console/)
│ │ ├── hub75/, webserver/, pubsubclient/ # Vendored libraries (see their VENDORED.md)
│ │ └── fonts/
│ ├── features/ # Everything that is not the device: one directory each
│ │ ├── pf_feature.h # The hook interface a feature implements
│ │ ├── pf_features.h # The dispatcher
│ │ ├── features.h # The default list (empty) and the escape hatch
│ │ ├── osc/ audio/ audio_in/ midi/ ble/ mqtt/ show/ weather/
│ │ └── README.md # What a feature is and the hooks it can answer
│ ├── console/ # The device's web pages as plain HTML — edit these
│ └── toolchain/ # PlatformIO extra_scripts (must stay sketch-relative)
├── bundles/ # Editions: two files each naming a feature set
│ ├── audio/ performance/ clock/ midi/ # features_local.h + overrides.h (+ env)
│ ├── build.sh # build the default, a named edition, or `all`
│ └── shelf.sh # stage a publishable image for the site
├── modules/ # The module toolchain's working folder, not an inbox — see modules/README.md
├── encoder_test/ # Standalone encoder diagnostic sketch
└── toolchain/ # Repo-level tooling: build a module or a pack, the console pages, the CI checks — see toolchain/README.md
```
`src/` holds the foundation that patterns build on; `features/` holds everything the device can do beyond being a panel with four knobs, and the core never names any of it — that rule, and why, is [`docs/EDITIONS.md`](../docs/EDITIONS.md). Patterns and the main sketch reference the helpers via `#include "src/core_*.h"`.
The foundation files are stateless utilities — no global state to coordinate, safe to include from any pattern.
## Foundation modules
Patterns should not duplicate trig tables, color converters, or noise functions. The foundation modules provide them once, shared across every pattern.
### `core_canvas.h` — PFCanvas
The single point of contact with the LED driver. Patterns write pixels into the canvas; the canvas pushes the frame to the HUB75 panel.
```cpp
PFCanvas::setPixel(x, y, r, g, b); // inside the pixel loop
PFCanvas::present(); // last line of draw()
```
Patterns must not call `dma_display->drawPixelRGB888()` directly. Global brightness, gamma, white balance, saturation, and any future post-processing live in `present()` — patterns that bypass the canvas miss those.
`present()` runs three post-processing steps before pushing pixels:
1. **Saturation boost** — pulls each pixel away from its Rec.601 luma in 8.8 fixed-point. Gray pixels are mathematically unchanged; saturated colors land closer to where the JS preview puts them. LED panels look washed out vs. a calibrated monitor, so a mild boost (default 1.10×) compensates.
2. **Per-channel gamma + white balance** — three 256-entry LUTs (one per channel) with the WB gain pre-multiplied into the gamma curve. HUB75 panels are linear PWM with unbalanced LED primaries (red is brighter per duty, blue is dimmer), so a single global gamma can't cover both correction needs. The pre-folded LUT means the inner loop pays the same cost as a single lookup.
3. **DMA push** — the vendored driver's `blitRGB888()` composes both scan halves and two adjacent columns per word, preserving the DMA control bits. Saturation and LUT work above are fused into that pass.
The current word-aligned blit measured **6.86 → 5.90 ms** on the bench's
128×64, 8-bit, 260 Hz configuration, with identical output. It needs no extra
framebuffer. See the [performance and recovery bench report](../docs/investigations/2026-09-firmware-runtime.md)
for the controlled comparison and its limits.
All five calibration values are tunable from `config.h` — see "LED panel calibration" below.
### Runtime notes (3.10): activation worker, memory admission, network maintenance, MIDI transport
The follow-up [runtime research](../docs/investigations/2026-09-firmware-runtime.md)
measures whole-loop delays, upload/load overlap, simulation warm-up and memory
policy. It separates tested improvements from experimental results and proposed
architecture work; its temporary profiler is not part of a normal build.
The resulting common-runtime implementation adds a small permanent `runtime`
snapshot to `/api/status`. Its whole-loop maximum includes work outside the
rendered-frame timer; `?resetTiming=1` returns the previous snapshot and begins
a new window. This is scheduling evidence, not a physical encoder-to-photon
measurement.
Pattern activation uses one reusable Core-0 worker with a **12,288-byte stack
reserved at boot**. Keeping that stack resident costs RAM between selections,
but avoids repeatedly finding a large contiguous block in a fragmented heap.
If it cannot be created, a later activation can retry; it never runs arbitrary
module setup on the render loop as a fallback. Boot restoration still uses
the synchronous path before interactive rendering begins. An allocation refusal
can be temporary while HTTP/UDP buffers are in use. The worker retries only
that failure class, at most five times with 1.5 seconds of total backoff.
Corrupt files and unresolved symbols fail immediately; the frame never waits
on the retry delay. `moduleMemory.loaderRetries` distinguishes recovered
admission attempts from a final `loadError`.
Storage mutations use conditional frame-boundary attempts. An unfinished
loader leaves the request pending while frames continue. Once quiescent,
activation is held during file changes, and later selections retain their
module path across catalog reordering. Catalog restoration starts an
asynchronous load. The single-core HTTP fallback replies busy when it cannot
complete the attempt inline. An open upload body prevents idle recovery from
starting a module against partially written storage; abort removes that partial
file and schedules restoration.
`src/core_module_memory.h` applies a 24,576-byte internal **byte-addressable**
service reserve to module allocations, including PSRAM-to-internal fallbacks.
Admission checks the relevant largest block and rechecks service RAM after an
allocation. Executable IRAM is not presumed interchangeable with byte RAM.
This protects against module allocations consuming the reserve; unrelated
system/feature allocations can still reduce free memory. Large data prefers
PSRAM; small data can remain internal. Module dynamic allocations keep their
16-slot, unload-as-a-group lifetime and add a configurable 4 MiB limit
(`PF_MODULE_RUNTIME_MAX_BYTES`); `moduleMemory` reports usage and refusals.
During loading, the loop waits out an 8 ms minimum period and running-mode
previews refresh at most every 64 ms unless their target/canvas changes.
Input and feature hooks still run each loop. Normal rendering has no new cap.
These policies apply to existing and future modules without editing pattern
code or changing the ABI.
The network worker's Wi-Fi/name maintenance hook is also serviced while the
HTTP parser waits, a conditional loop request is pending, or a page-body
socket is backpressured. Nonblocking sends retain the existing 20 s stall /
120 s total budgets. No HTTP re-entry or extra task is involved. Filesystem
operations and other synchronous library calls can still delay maintenance;
`netMaintenance.maxGapMs` exposes those gaps.
The Audio edition's MIDI feature additionally owns a separate 4,096-byte
Core-0 transport worker. A display frame can be long enough to overflow the
SDK's six-datagram UDP mailbox when four independent CCs arrive together.
The worker parses and sends RTP-MIDI between frames, while fixed queues hand
events to/from the frame task. Their 1,536 bytes of payload storage live in
PSRAM; queue indices and synchronization remain internal. The worker never
changes the parameter bus or a pattern directly. If storage or worker creation
fails, frame polling remains available and `midi.rxDetail.worker` reports
false. This implementation lives entirely in `features/midi/`.
See the [implementation and MIDI follow-up report](../docs/investigations/2026-09-firmware-runtime.md)
for measured before/after results and remaining limitations.
### `core_math.h` — PFMath
```cpp
PFMath::buildSinLUT(); // call from setup() — idempotent
PFMath::fastSin(angle); // ~5x faster than sinf in pixel loops
PFMath::fastCos(angle);
PFMath::fract(x); // x - floor(x), without calling floorf
PFMath::ifloor(x); // floor as an int; floorf() is a libm CALL here
PFMath::floorF(x); // floor as a float, same reason
PFMath::clamp(x, lo, hi); // fminf/fmaxf are libm calls too
PFMath::clamp01(x);
PFMath::jsMod(x, m); // JavaScript's x % m (sign of x), no fmodf
PFMath::lerp(a, b, t);
PFMath::approxLength(x, y); // 6.8% accurate sqrt(x*x + y*y)
```
The sin LUT is 4 KB (1024 entries, ~0.35° resolution) and shared. Do not build your own.
**`floorf`, `fminf` and `fmaxf` are function calls on this target**, not instructions - GCC emits a call into libm at every use, at every optimization level. That is what `ifloor`, `floorF`, `fract`, `clamp` and `clamp01` exist for: they truncate and correct, or compare, which the FPU does in one instruction each. Removing a single `floorf` from a per-pixel path measured *smaller* object code as well as faster, because the call sequence costs more than the inline does.
`approxLength` is an octagonal sqrt approximation — 6.8% error (measured by `toolchain/check_math.py`, which pins it; the 0.375 coefficient is the shift-friendly one, not the minimax one), no `sqrtf` in the pixel loop. Use it only when distance is a **secondary** signal. If distance IS the visual structure of the pattern (radial ripples, concentric rings, vortex centers, anything that uses `1/dist` for amplification), use real `sqrtf` instead — the octagonal contour shows up as visible polygonal artifacts in those cases.
### `core_color.h` — PFColor
```cpp
PFColor::hsvToRgb(h, s, v, r, g, b); // h is 0..1 (not degrees)
PFColor::ColorStop ramp[] = { {0.0f, 0,0,0}, ... };
PFColor::sampleRamp(ramp, count, t, r, g, b); // POSTERISED - flat bands, no divide
PFColor::sampleRampLerp(ramp, count, t, r, g, b); // gradient - one divide per sample
```
### `core_noise.h` — PFNoise
```cpp
PFNoise::perlin2D(x, y); // NOT normalised: runs to about +/-1.51
PFNoise::valueNoise2D(x, y); // 0..1
PFNoise::cellHash(gx, gy); // 0..1
PFNoise::cellHash(gx, gy, seed); // seed decorrelates, measured
PFNoise::fractal2D(x, y, octaves, roughness);
```
The 512-byte permutation table is shared. Do not duplicate it.
### `core_ota.h` — PatternflowOta
ArduinoOTA wrapper for wireless flashing. The main sketch only needs:
```cpp
PatternflowOta::begin(); // in setup() — connects Wi-Fi if not already up
PatternflowOta::handle(); // first line of loop() — UDP poll, ~free when idle
```
Shares Wi-Fi credentials and connection with OSC (if both are enabled, the connection is reused). When `PF_OTA_ENABLED` is 0 everything compiles to a no-op. See the [OTA Updates](#ota-updates-for-developers) section below for the user-facing workflow.
### `core_improv.h` — PatternflowImprov
Self-contained [Improv-Serial](https://www.improv-wifi.com/serial/) implementation so the **browser flasher** (ESP Web Tools, behind the website's "Flash" button) can set Wi-Fi over USB right after flashing — no rebuild, no `patternflow_secrets.h`. The main sketch only needs:
```cpp
PatternflowImprov::begin(); // in setup() — announces it speaks Improv
PatternflowImprov::handle(); // in loop() — drains Serial, drives provisioning
```
The flasher sends the SSID/password over serial; `core_wifi.h` stores them in NVS (separate `pf_wifi` namespace) and uses them in preference to the built-in `PF_WIFI_SSID/PASS` placeholders on every boot. Shares the debug USB Serial — the host parser scans for the `IMPROV` header and ignores `println()` noise. Compiled in only when Wi-Fi is actually used (one of OTA/OSC/audio enabled) and `PF_IMPROV_ENABLED` is 1; otherwise a no-op. Set `#define PF_IMPROV_ENABLED 0` in `patternflow_secrets.h` to drop it.
## Patterns
The pattern list has two tiers:
- **Presets** — the curated showcase in `presets/`, compiled into `firmware.bin`.
Pattern 1 is always `Origin`. They work whether or not the filesystem mounts,
and switching between them is instant.
- **Modules** — `.pfm` files on the FATFS partition, discovered at boot and
appended after the presets. This is how new patterns normally arrive now: a
~3–22 KB file installed over Wi-Fi at `http://patternflow.local/patterns`,
**no reflash, no reboot**. See [Loadable pattern modules](#loadable-pattern-modules-pfm).
Both tiers share the same pattern shape — a namespace with:
- `NAME`
- `KNOB_LABELS`
- `setup()`
- `update(float dt, const InputFrame& input)`
- `draw()` — draws via `PFCanvas::setPixel(...)` and ends with `PFCanvas::present();`
and the same foundation (`PFCanvas`/`PFMath`/`PFColor`/`PFNoise`), so a pattern
written for one tier ports to the other mechanically. The everyday way to make
one is the Pattern Lab — [`PATTERN_GUIDE.md`](../PATTERN_GUIDE.md) — which
generates conforming C++ from a JavaScript pattern and installs it over Wi-Fi;
[`CUSTOM_PATTERNS.md`](CUSTOM_PATTERNS.md) is the hands-on route.
To add a **preset** (rare — curated set): copy `_TEMPLATE.h` into
`presets/preset_<name>.h` and add one `PATTERN_ENTRY(...)` line in
`pattern_registry.h`. To add a **module** (the usual way): see the next section.
## Loadable pattern modules (.pfm)
Patterns no longer have to be compiled into the firmware image. A pattern can
be built as a relocatable Xtensa ELF module, copied to the device over Wi-Fi,
and it appears in the list immediately.
```
firmware/modules/<slug>/pattern.cpp ← freestanding pattern source
│ python firmware/toolchain/build_module.py firmware/modules/<slug>
▼
<slug>.pfm (+ <slug>.json sidecar) ← 3–22 KB relocatable ELF
│ drag onto http://patternflow.local/patterns (several at once)
▼
FATFS /patterns/<slug>.pfm → loaded on selection in ~5–11 ms
```
The design comes from Simone Majocchi (@SimonePDA), who proved the whole idea
in a working fork: a frozen C ABI between host and module, a linker script
that collapses each module to four sections, and an on-device relocator.
**How it works, briefly.** The firmware exposes a versioned function table
(`abi/pf_abi.h` — framebuffer, present, alloc, log, millis, polar tables,
libm hooks). A module is compiled freestanding against `abi/pf_module.h`
only — never against Arduino or the HUB75 driver — and partially linked with
`toolchain/module.ld` so all code lands in one contiguous `.text` with
literals inside it. At load time `core_module_loader.h` reads the ELF from
FATFS, allocates executable internal RAM, applies `R_XTENSA_32` relocations,
resolves libm/libgcc against the host's own symbols, runs `.init_array` (so
C++ global constructors work), syncs the instruction cache, and calls the
module's entry point. Switching away unloads it; one module is resident at a
time.
**Converting an existing pattern header:**
```bash
python firmware/toolchain/port_preset.py firmware/patternflow/presets/preset_origin.h
python firmware/toolchain/build_module.py firmware/modules/origin
# outputs land in firmware/patternflow/data/patterns/ — upload from /patterns
```
The port is mechanical (drop firmware includes, add `#include "pf_module.h"`,
append `PF_REGISTER_PATTERN(Namespace)`); the body is copied verbatim because
the module SDK provides the same `PFCanvas`/`PFMath`/`PFColor`/`PFNoise`
surface — the math headers are literally the same files, included with
`-DPF_MODULE_BUILD`.
**Install paths**, easiest first:
- Community cart → "Send over Wi-Fi": opens `/patterns?src=<modules-url>`, and
the page fetches and installs every file itself. Nothing downloaded.
- Drag any number of `.pfm`/`.json` files onto `/patterns` — a visible queue
uploads them one by one with progress, retries, and per-file results.
- `curl -F "module=@slug.pfm" http://patternflow.local/api/patterns` for scripts.
**Costs and limits, measured on hardware** (v3.2, 2026-07, 128×64, esp32 core 3.3.8 build — a core 2.x build has more heap and renders faster; see [How to measure this](#how-to-measure-this-before-reading-it)):
| Property | Measured |
|---|---|
| Switch latency (7.5 KB module) | 6.0 ms = read 4.4 + relocate 0.6 + setup 1.0 |
| Switch latency (22 KB module) | 10.9 ms — still under one 60 fps frame |
| Runtime speed vs the same source compiled in | **~20 % slower** (Origin: 53.4 → 43.5 fps) — the cost of relocatable code (`-mlongcalls`); not fixable by `-O2` (~2 %) or memory placement |
| Comfortable module size | ~32 KB of statics on a core 2.x build — measured: 0–64 KB of static tables all render at the same 62 fps, they just spend heap. (The old "≲ 8 KB, then `.rodata` spills to PSRAM and ~14 fps" figure was the core 3.x build's 7.7 KB largest block talking; see [Required board package](#required-board-package).) |
| Panel size | Baked in at build (`-DPF_PANEL_W/H`); the loader rejects a mismatch |
| ABI | `PF_ABI_VERSION` must match; bump it on ANY layout change in `pf_abi.h` and rebuild every module |
If a module fails to load, nothing else is affected — the presets and other
modules keep working, the panel shows a `PATTERN FAILED` screen with the
reason, and `/api/status` reports it as `loadError`. Check `loadError` first
for any "this pattern is broken" report — it turns a vague complaint into a
specific one.
`unresolved symbol: X` used to be the usual cause, and it is worth knowing
what it cost before it was closed: the table carried `tanhf` but not `coshf`,
and since sech(x) = 1/cosh(x) is the closed form of a soliton, an entire
genre of wave patterns failed to load in a way that read as "too heavy for
the board". As of **v3.5.2** the table was audited in bulk — every shipped
module's demand plus a synthetic probe of the whole libc/libm surface
(hyperbolics, erf, the rintf family, 64-bit integer helpers, qsort/strto*,
string/ctype, an atexit shim) — and `build_module.py` now checks every
`.pfm` it links against the loader's own table, so a pattern the device
would refuse fails **on the build machine with the missing names spelled
out**. If a new name ever does surface, the fix is still one
`PF_HOST_SYMBOL` line in `resolveHostSymbol()` — add it there and the build
checker learns it automatically, since it parses that header.
**Pattern names are UTF-8.** "Dynamic Moiré" and "Poincaré Sphere" are real
entries in the library, and an accent used to break three layers at once: the
toolchain scripts printing a name on a cp949 console, a loader guard that
tested names for printable ASCII and called anything else a relocation bug,
and the panel's 5×7 ASCII font. Names stay UTF-8 in the ELF and in every JSON
API; only the panel folds them (`asciiFold()` in `patternflow.ino`). Any new
code that touches a name must assume UTF-8.
### How many can you install
Installing costs nothing at runtime. The registry allocates its arrays at full
capacity on boot, in PSRAM, whether or not the modules exist — so five
installed modules and 128 installed modules use exactly the same RAM. Only the
**resident** module costs internal RAM, and unloading returns all of it
(measured: 4,548 B free with a module resident → 11,692 B on a preset).
| Limit | Value | Binding? |
|---|---|---|
| Installed modules | **128** (`MAX_MODULE_PATTERNS`) | The only real cap, and it is a UX choice — 136 B of PSRAM per slot, 17 KB total |
| Storage | ~1,500 modules (10.2 MB partition, 5.9 KB median) | No — 12× the count cap |
| Per-module RAM | ~32 KB of statics comfortable on a core 2.x build (0–64 KB measured at identical fps; the old ≲ 8 KB figure was the core 3.x heap talking) | Heap only — frame rate does not move |
| Concurrent modules | 1 | By design — one pattern is selected at a time |
A large module only costs while it is the selected pattern; it has no effect
on the presets or on any other module. On the core 3.x build a resident
module dropped `heapLargest` to ~3 KB — which is what made opening a console
page pause the pattern; on the core 2.x release build the same module leaves
~65 KB and the console stays comfortable (the pause-on-open mechanism remains,
see the constraints section below).
Measured across the real 42-pattern community library: median `.pfm` 5,924 B,
largest 17,512 B, smallest 4,612 B, 281 KB for all 42 together.
### How to measure this, before reading it
`heapLargest` is not a property of a build. It decays as the device serves
HTTP, in steps of exactly 2,048 B — polling a build eighteen times walks it
down ~4 KB — so **a firmware sampled often reads worse than one sampled
rarely**, and two numbers taken over different windows do not compare.
Take it this way, or do not quote it:
> flash → `POST /api/wifi/reboot` → wait ~80 s → **one** read of `/api/status`
Two names that mean different things, and have already been mixed up here:
- **Flash.** The toolchain's figure is program storage (1,412,457 = 44.9 % of
the 3,145,728 B partition). The `.bin` on disk is ~360 B larger — header,
padding, hash. Say which one you mean.
- **RAM.** The build's `92,920` is static RAM *used*, 28.4 % of 327,680. Free
internal heap is a different number and only the running device knows it.
And the rule the rest of this section exists to serve: **a heap number
travels with its build, its procedure and its date, or it is not evidence.**
The figures below are v3.5.2 on a core 2.x build and are quoted as history —
the [September 2026 runtime report](../docs/investigations/2026-09-firmware-runtime.md)
has the 3.10 numbers, taken under the protocol above. In 2026-08 the `86,004` below was copied into five
documents as a measurement of something else entirely.
### Current state (v3.5.2, measured on hardware, core 2.x build)
| | |
|---|---|
| Frame time, preset (Origin) | ~16,270 µs = **61.5 fps** |
| Internal heap, preset resident | 96,324 B free, largest block 86,004 B |
| Internal heap, module resident (Muybridge 35mm, 29 KB `.pfm`) | 76,668 B free, largest block 65,524 B — 48.5 fps |
| PSRAM | 8.33 MB idle |
| Flash | 1,148,960 B = **37 %** of the app partition |
| Globals | 84,660 B = **26 %**, 243 KB left for locals |
| Pattern storage | 131,072 / 10,235,904 B with 13 modules installed |
| Leak check | 48 consecutive page loads: **−8 B** net, largest block unchanged *(v3.1.0 measurement; not re-run since)* |
The v3.1.0 / core 3.x numbers this table used to carry — 53.4 fps, 11.7 KB
free, 7.7 KB largest block — are what the constraints section below calls the
other build. Anyone comparing a measurement against this table should first
check which core their firmware was built with.
## Adding features — the constraints that matter
Read this before adding anything to the firmware. Every item here was learned
the expensive way, with measurements to back it.
### Internal RAM is the budget. Everything else is roomy.
The ESP32-S3 has 512 KB of internal SRAM, but what a feature can actually
claim is far smaller:
```
≈320 KB Arduino data region (of 512 KB SRAM; rest is cache/ROM/RTOS)
−92 KB this firmware's globals (24 KB canvas, fonts, module state…)
=213 KB free heap at boot ← measured, core 2.x
−150 KB HUB75 DMA framebuffers (see below)
−49 KB Wi-Fi socket buffers, HTTP, OSC, mDNS…
≈ 15 KB steady-state free internal heap
```
**That last line is a build choice, not a hardware limit.** The figures above
are the core 3.x picture. Core 2.x gives back roughly 71 KB that IDF 5 takes
before `setup()` even runs, so the same firmware idles at **~98 KB free,
90 KB largest block** — see [Required board package](#required-board-package).
Build on core 2.x and the numbers below stop being the constraint they were.
Build on core 3.x and they are real, so know which one you measured on before
concluding a feature is too expensive.
Below roughly **10 KB free**, the web console starts failing in a maddening
way: every endpoint returns its status line and then hangs, while Wi-Fi, OSC
and OTA look perfectly healthy. A 7 KB static array added during development
took down four unrelated pages at once. So:
- **Static buffers are guilty until proven innocent.** If data is written at
boot and read occasionally — lists, tables, names — allocate it from PSRAM
(`heap_caps_calloc(1, size, MALLOC_CAP_SPIRAM)`). There are **8 MB** of
PSRAM sitting idle; the pattern registry lives there for exactly this
reason. Reserve internal RAM for hot loops and DMA.
- Check `/status` (or `/api/status`) after your change: it reports free
internal heap, largest block, and PSRAM live from the device.
### Why HUB75 owns 150 KB and it cannot move
HUB75 panels have no framebuffer of their own and only understand on/off per
pixel. The driver therefore streams the whole frame continuously (the panel
lights 2 rows at a time) and achieves 256 brightness levels by re-sending the
frame as 8 binary-weighted bit-planes (BCM). That waveform is what DMA plays
out autonomously:
```
128 cols × 32 row-pairs × 2 B = 8 KB per bit-plane
× 8 bit-planes = 64 KB per frame buffer
× 2 (double buffering) = 128 KB (+ descriptors ≈ 150 KB)
```
It must be internal RAM — PSRAM's latency jitter glitches the 15 MHz stream.
Shrinking it costs image quality directly (6-bit color = banding in exactly
the gradients these patterns live on; single buffering = tearing). This is
the standard tax every ESP32-driven HUB75 project pays; in exchange, refresh
costs zero CPU and the render loop effectively owns a core.
### The Arduino WebServer will hurt you. Known landmines:
> **Writing a client rather than a route?** The endpoints, their JSON shapes and
> the rules that follow from this section are specified in
> **[docs/rest-api.md](../docs/rest-api.md)**. Build against that file; this one
> explains *why* the server behaves as it does, which is what you need when
> adding to it.
The console shares **one** synchronous, single-client `WebServer` on port 80
(owned by `core_audio_ws.h`; other features attach routes to it). Learned so
far, each confirmed by A/B on hardware:
1. **A multipart POST with a URL query string gets an empty reply.** Send
flags as form fields instead (`fd.append('last', …)`), never `?last=1` on
an upload.
2. **Never do heavy work inside a request handler.** A FATFS rescan + module
reload inside the upload handler crashed the device mid-batch — and the
crash corrupted the FAT. Handlers set a flag; `tick()` functions called
from `loop()` do the work ~150 ms later (see `core_patterns_http.h`).
3. **Never force-close connections from the server** (`client().stop()`).
Each server-initiated close parks a TCP pcb in TIME_WAIT; after a couple
of upload batches every multi-chunk transfer died until reboot.
4. **Rapid sequential uploads flake ~1 in 12** even when healthy (dead reply
after the file was stored). Client pages pace files ~350 ms apart and
retry each file twice; re-uploading is harmless (same bytes, same name).
5. **Never auto-format the filesystem on a failed mount.** A crash mid-write
once corrupted the FAT and the "helpful" auto-format erased every
installed module. Mount failure now means presets-only; formatting is an
explicit button (`POST /api/patterns/format`).
6. **A response larger than ~5.6 KB needs the heap a loaded module is
holding.** This one looked for a long time like a slow "leak" that
degraded over hours; it is not. Measured on one device, one minute apart:
```
module resident internal heap 4,932 /patterns truncated at 5,633 B
after a 10 s stall
module unloaded internal heap 11,852 /patterns whole (15,903 B), 0.43 s
```
5,633 B is four TCP segments — one bufferful. Under a starved heap the
one-shot `send()` fills that and gives up, and the client waits out the
timeout holding half a page. The HTML renders, its script is cut
mid-statement and never runs, and the console looks blank while every API
underneath answers fine.
Until 3.6.3 the console therefore **paused the pattern**: opening a page
evicted the module and restored it after 25 s of silence. Since 3.6.3 the
page sender streams PROGMEM in small slices under a 5-second budget, the
pages are gzip-compressed, and the core-2 builds have the heap for it, so
a page no longer evicts anything; `status.consolePaused` now means only
that a pattern-install batch is in progress ([`docs/rest-api.md`](../docs/rest-api.md)).
The memory wall below is still real, and is why pages stay small.
**This is a memory wall, not a pacing problem.** It is tempting to read
the 10 s stall as impatience — `NetworkClient::write()` really does send
with `MSG_DONTWAIT` and give up after `WIFI_CLIENT_MAX_WRITE_RETRY` (10)
non-writable selects of 1 s each, which is where the ten seconds comes
from. Rewriting the send path to be patient does not help, because the
number that matters is `heapLargest`: **3,060 bytes** with a module
resident. lwIP cannot allocate the pbufs, and no amount of waiting
creates them.
Three fixes were tried on hardware and **do not work**; do not retry them:
- Spilling module data sections to PSRAM to spare internal RAM — reboots
the device the moment a module is selected.
- Chunked transfer encoding — delivers *less*, not more (one 1 KB chunk,
or nothing), with or without pacing between chunks.
- 512-byte slices with a wall-clock deadline instead of a retry count
(`core_http_send.h`, reverted in bb8e52c) — delivers **3,072 B**, worse
than the 5,633 B it replaced, and still takes the full 10 s. It *is*
~3× faster than `send_P` on a healthy heap, which is exactly the trap:
it measures beautifully in the condition that was never the problem.
An async server (ESPAsyncWebServer or esp-idf httpd) would lift the
one-client and upload-pacing limits, but it shares this same lwIP heap —
do not expect it to make a 16 KB page deliverable on a 4.5 KB heap. The
only real levers on page delivery are using less internal RAM per module
or shrinking the pages.
### Adding a console page
The pages are plain HTML in [`patternflow/console/`](patternflow/console/); edit them there, preview with `console_serve.py`, and `console_pages.py build` splices them into the `*_index.h` headers — the whole workflow is [`patternflow/console/README.md`](patternflow/console/README.md). A page that belongs to a feature travels with the feature (`navPath`/`navLabel`/`navDesc` in its descriptor); the core never names it. Two rules that the README does not repeat:
- **Syntax-check the page's JavaScript before flashing**: extract the `<script>` body and run `node --check` on it. A single stray newline in a string once shipped a page whose script never ran — the page rendered but showed nothing, which reads as "the device is broken".
- **Never leave the render loop with nothing to draw.** If a screen decides not to paint, set `frameDrawn = false`; flipping an unpainted buffer shows a torn leftover frame, which every tester reads as "the pattern is broken".
### What is cheap and what is expensive here
Cheap (fits the platform well):
- **Wired OSC over USB-C** — the S3's native USB does CDC serial out of the
box; OSC 1.1 over SLIP is a standard TouchDesigner/Max speak. Reuses the
existing OSC message parser with a different transport. Near-zero RAM,
independent of Wi-Fi entirely. A very good first contribution.
- More device pages, more OSC addresses, more module-SDK helpers (add the
symbol to the host table + bump nothing, if it's a new function on
`PFHostAPI` — that's an ABI bump).
- Anything whose cold data can live in PSRAM.
Expensive (think twice, or push to "Remote compute"):
- TLS clients/servers (tens of KB of internal RAM per connection).
- Many concurrent WebSockets or HTTP clients (single-threaded server today).
- Anything needing large always-resident internal-RAM buffers — that budget
is 15 KB, total, shared with everyone.
## Configuration (`config.h`)
All hardware-specific pins and limits are centralized in `config.h`.
- **Panel Selection:** `PANEL_PROFILE` picks the driver-init path — `PANEL_STANDARD` (no init; the default and what the browser flasher ships), `PANEL_HIGHREFRESH` (FM6126A), `PANEL_FM6124`, `PANEL_ICN2038S`, `PANEL_MBI5124`, `PANEL_DP3246`. It expands to the `HUB75_DRIVER` value `core_display.h` passes to the library. **Don't set it by part number — leave it at `PANEL_STANDARD` unless the panel comes up completely dark.** The init sequence only writes a brightness register and an output-enable bit; many panels ship with those already usable, and the reference panel's FM6124 runs fine without it. A panel that genuinely needs a different profile needs one custom build. Full buyer's guide: [docs/panel-compatibility.md](../docs/panel-compatibility.md).
- **Pin Mapping:** Adjust the `R1_PIN`, `ENC1_A` etc. if you are not using the official Patternflow PCB.
- **Hardware Settings:** `INVERT_ENCODER` can be toggled depending on whether you mounted your encoders on the front or back of the PCB. `DEFAULT_BRIGHTNESS` controls the initial matrix brightness.
### LED panel calibration
LED panels render the same RGB triplets differently than a calibrated monitor — the LED primaries are at different wavelengths than sRGB phosphors, red LEDs are brighter per PWM duty than blue, and the panel reads washed-out next to a monitor showing the same frame. The defaults are the ones that ship in `config.h` (saturation tuned by eye on 2026-08-09 against the Calibration preset, on the maintainer's panel; white balance is identity since 2026-09-17, when the bit-plane weights were made binary and the earlier trim turned out to have been correcting the driver rather than the panel); every value is `#ifndef`-guarded, so a panel can override any of them from `patternflow_secrets.h`:
```cpp
#define LED_GAMMA_R 1.0f // the pow-LUT already handles perceptual response;
#define LED_GAMMA_G 1.0f // extra gamma here only made the panel darker
#define LED_GAMMA_B 1.0f
#define LED_WB_R 1.000f // identity: the reference panel reads neutral uncorrected.
#define LED_WB_G 1.000f // Judge a trim on the GRAYS screen, not on white alone
#define LED_WB_B 1.000f
#define LED_SAT_BOOST 1.62f // strong: matches design intent on real patterns
```
Tune live from `/api/display` (the `/status` page has the sliders), then land the converged numbers in your secrets file. The comments in `config.h` above each value record how they were arrived at.
### Refresh rate (anti-flicker for video)
`core_display.h` sets the refresh floor, and the vendored driver lands the panel on **exactly 300 Hz**:
```cpp
mxconfig.i2sspeed = HUB75_I2S_CFG::HZ_15M; // pixel clock — see below, this is 16 MHz
mxconfig.min_refresh_rate = 240; // refresh floor (shipped: exactly 300 Hz)
mxconfig.latch_blanking = 2; // brightness uniformity
```
> **The library's clock enum names are wrong.** `HZ_15M` is `16000000` and `HZ_10M` is `8000000` — `HZ_8M` and `HZ_10M` are the same value, and there is nothing between 8 and 16 MHz. Read the enum, not the name, and if you are hunting harmonics on a spectrum analyser the fundamental is at **16 MHz** (so 96, 112, 128, 144, 160 MHz…), not 15.
A HUB75 panel lights one row pair at a time: each row is on for 1/32 of a refresh and dark for the rest — a short pulse, not a glow. A phone's rolling shutter exposes each sensor line for the same length of time but starting a little later than the one above, so each line collects a whole number of those pulses. If the exposure is an exact multiple of the refresh period, every line collects the same number and the picture is clean. If it is not, some lines get N pulses and some N + 1, and that is the banding you see on video — its contrast is about 1/N, and how much of the picture it covers depends on how far the exposure is from a whole number of periods. A faster refresh makes the bands fainter and *more numerous*; it does not make them go away.
Exposure times are not arbitrary, though. To dodge mains flicker a phone pins them to 1/50, 1/100, 1/25 (50 Hz countries) or 1/60, 1/30 (60 Hz countries), and **300 Hz is the lowest rate all of those are whole multiples of**. The shipped chain would naturally run at 325.5 Hz — nearly the worst case for 1/50 (6.51 periods) and 1/60 (5.43) — so the driver ends every frame with 4,181 blank clocks, output off, to make it 16 MHz / 300 = 53,333 clocks exactly (`PF_TARGET_REFRESH_HZ` in `src/hub75/`, 0 to switch off; it never pads below `min_refresh_rate`). About 8% of the light, and no CPU: refresh runs on the ESP32-S3's DMA hardware in parallel with everything else.
What 300 Hz cannot fix: **1/120 and faster, and slow motion.** At 1/120 an exposure is 2.5 periods — the worst case — and the rate that would cover it, 600 Hz, is out of reach of a 16 MHz clock at 8 bits. When filming: lock the shutter at 1/60 or 1/50 (a manual camera app, or pro mode), or turn the panel *down* so the phone lengthens its exposure; lock white balance and focus; switch HDR and night modes off.
The trade-off is brightness. A row is sent as a chain of bit-plane buffers, and the vendored driver picks the brightest chain that still clears `min_refresh_rate` (`pfChainExtraPasses` in `src/hub75/`, and `VENDORED.md` beside it for why the chain is not upstream's). Every scheme is exactly binary — colour depth is never what gets sacrificed — but a shorter chain keeps less of each row lit. **Both settings move together — you cannot lower one without deciding about the other.** On the reference panel (128×64, 8-bit):
| `i2sspeed` | `min_refresh_rate` | Chain | Row lit | Actual refresh |
|---|---|---|---|---|
| 16 MHz | 240 *(shipped)* | scheme 0, 12 buffers, padded | 61% | 300 Hz exactly |
| 8 MHz | 240 | scheme 2, 8 buffers | 25% — **dimmest** | ~244 Hz |
| 8 MHz | 200 | scheme 1, 9 buffers | 44% | ~217 Hz |
| 8 MHz | 150 | scheme 0, 12 buffers — same as shipped | 66% | ~163 Hz |
So if you drop the clock, **drop `min_refresh_rate` with it** — halving the clock while leaving 240 in place is the worst of both worlds, giving up most of the panel's light to claw back a refresh rate you were not going to keep anyway. Banding on *video* of the panel is a matter of shutter against refresh, as above — at 8 MHz the frame is already longer than 1/300 s, nothing is padded, and the rates in the table are what you get. (The 8 MHz rows are computed from the chain lengths, not re-measured since the chain changed; the 16 MHz row is what the panel reports.)
> **Lowering the panel clock is also an EMI knob**, and it has been measured. 8 MHz improves Wi-Fi latency on the reference unit (median 15 → 4 ms) and one contributor found it was the difference between a usable and an unusable radio on their board — but at any `min_refresh_rate` that preserves colour depth it bands on video, so it is not shipped. Full measurements, the parts that cannot be adopted (do **not** raise the Wi-Fi TX power — it is a conformance setting), and what to try first if your unit's Wi-Fi is bad: [docs/investigations/2026-08-the-panel-clock-and-the-wifi-radio.md](../docs/investigations/2026-08-the-panel-clock-and-the-wifi-radio.md).
## Controls
The four rotary encoders control pattern-specific parameters. Each pattern exposes its own labels through `KNOB_LABELS`.
For the original defaults:
- **Encoder 1:** Hue
- **Encoder 2:** Speed
- **Encoder 3:** Mode/Preset
- **Encoder 4:** Frequency
### Longpress actions
- **Encoder 1 longpress (≥1s)** — enter/exit global brightness mode. While active, K1 rotation adjusts panel brightness (5–255, ~5 per detent), the active pattern does not see K1 input, and a "BRIGHTNESS XX%" overlay shows the current level. Exits on a second longpress or after 5 seconds of idle. Value persists across reboots via NVS.
- **Encoder 2 longpress (≥1s)** — enter/exit the NETWORK screen (portrait status view: Wi-Fi state, local IP, OSC on/off and its remote host/port, audio-react on/off). Inside the screen, settings are toggled by **knob rotation, not clicks** — right = ON, left = OFF — so that holding K2 to leave can't flip anything on the way out:
- **turn K2** → OSC send/receive on/off (persists in NVS, so the device reboots into the same state). Wi-Fi stays connected either way; the toggle only enables/disables traffic. If the firmware was built with `PF_OSC_ENABLED 0` the row still shows but the toggle is inert ("REBUILD WITH PF_OSC_ENABLED=1").
- **turn K3** → audio-react on/off (also persisted).
- **turn K1** → sleep (see [Sleep mode](#sleep-mode) below). Either direction, unlike the toggles above: the screen is about to go dark, so "right = on" has nothing to mean.
- **turn K4** → hand off to the UPDATE screen, which arms the `/update` endpoint (the arming *is* the security model — see `core_web_update.h`).
Exits on a second K2 longpress, a **K2 click**, or after 8 seconds of idle.
- **Encoder 3 longpress (≥1s)** — enter/exit the KNOB MAP screen: it shows which physical knob is which number (front view: K1 top-right, K2 top-left, K3 bottom-right, K4 bottom-left), and turning any knob lights its digit green so each one can be verified without leaving the screen. Knob input is swallowed while it's up, so the pattern underneath never sees it. Exits on a K3 click, a second K3 longpress, or after 8 seconds of idle.
- **Encoder 4 longpress (≥1s)** — enter/exit pattern SELECT mode. In SELECT mode, K4 rotation moves the highlight through the list — three detents per pattern, so a hand does not overshoot — and the panel shows each pattern's **thumbnail** (the frame it last drew, kept from the last time it ran); the highlighted pattern loads once the knob has rested for a third of a second — on the other core, so the knob keeps answering while a heavy pattern's setup runs — then runs live behind the overlay. A pattern that has never run shows nothing but its name until it has. Longpress again to confirm — a choice the knob was still resting on loads right then.
Thumbnail ownership follows the frame boundary too: the loop owns the PSRAM
cache, and the existing network task reads/writes one immutable job at a time.
Capturing a picture copies pixels; it opens no file. A completed read is adopted
between frames, and deletion/format invalidates jobs by generation. The optional
cache and its one 16 KB work buffer use PSRAM exclusively; allocation failure
leaves the name-only preview. No extra task stack is reserved. This separation
does not shorten a module's own `setup()`; the asynchronous loader still handles
that wait. `/api/status.thumbs` separates capture cost from background I/O cost.
### One detent, one step
Knob deltas are linear: a pattern sees exactly the number of detents that were turned, however fast the turn was.
There used to be a fast-spin multiplier here (×2 to ×5 as the gap between detents shrank), meant to let one knob sweep a wide range quickly. It was removed after testing against a linear build, because it made the knobs unpredictable on exactly the parameter that most needs landing on a value — Origin's Mode knob picks a discrete preset, and a quick turn skipped five at a time. OSC had already been routed around the multiplier for the same reason, which was the tell.
**If a pattern's range feels slow to cross, raise that pattern's own step constant** (`d * 10` → `d * 25`). Linear and predictable beats a curve guessing at intent.
### Short press (per-pattern, opt-in)
There is no global short-press handler. Each pattern decides what `K1..K4 short press` does for itself, by reading `input.btnPressed[i]` inside its `update()`. The built-in patterns (`Origin`, `Wave Saw`) use short press to reset the corresponding parameter to its default. A pattern that does not handle `btnPressed` — most of the curated presets — simply ignores short presses.
When you generate a new pattern from the Live Editor, the conversion prompt does not force a particular short-press convention — if you want one, either ask for it in the prompt ("K1 short press resets hue") or add the line by hand in `update()`.
## Sleep mode
`src/core_sleep.h`. The panel goes dark and the board idles, without anything being unplugged, and it stays **on the network** the whole time.
That last part is the design decision. `esp_deep_sleep` would take the draw to microamps, but it takes the radio with it, and then the only way back is a button — which is exactly what "switch the lights off from the sofa" needs not to be true. So sleep here means:
| | |
| :--- | :--- |
| **Panel** | framebuffer blanked, brightness 0, HUB75 DMA transfer stopped |
| **Wi-Fi** | modem sleep — still associated, wakes on its DTIM beacon |
| **CPU** | 80 MHz, the floor at which the radio still works |
| **`loop()`** | returns early and yields 20 ms; no render, no flip, no power clamp |
Stopping the DMA transfer is what makes this worth more than blanking. The LEDs are most of the draw and blanking collects that, but with the transfer running the panel's driver ICs are clocked at 15 MHz forever, lit or not. `core_power.h` puts that idle floor at ~550 mA against ~1900 mA for a bright pattern.
The yield is not politeness either: without it the sleeping loop polls sockets flat out and the core never idles, so neither modem sleep nor the IDF's clock gating gets a chance.
### Entering and leaving
| Enter | Leave |
| :--- | :--- |
| the **On / Sleep** switch on the console home page (`/`) | the same switch |
| NETWORK screen (hold K2) → turn K1 | any **physical** knob turn or button press |
| MQTT `<prefix>/sleep` ← `1` / `on` / `true` / `sleep` / `toggle` | same topic ← `0` / `off` / `false` / `wake` |
| `POST /api/sleep` ← `on=1` (also `on=toggle`) | `POST /api/sleep` ← `on=0` |
| | a firmware image starting to arrive (web update or `espota`) |
"Physical" is meant literally: waking checks the raw encoder counters, not `input.knobDeltas`, because remote OSC/MQTT/audio deltas are merged into that field by the time the input frame exists. A show still streaming knob values at a sleeping panel must not switch the lights back on. Input is ignored for the first 800 ms so the second detent of the K1 turn that started the sleep doesn't end it.
Sleep is never entered while an image is being written to flash, and a device already asleep wakes when one starts arriving — whoever is flashing wants the UPDATE screen anyway.
The state is **not** persisted: a device unplugged while asleep boots awake, because a panel that stays dark after you plug it in reads as broken.
### Waking gives the pattern back
The one thing that still evicts the resident module is a pattern-install batch (a console page has not since 3.6.3). So a wake — from *any* of the three sources, not just the web switch — asks for the pattern back via `PatternflowPatternsHttp::requestReload()`, the same path the console's Play Now button uses, and gets it whether or not something was holding it. The reload happens at the frame boundary from `loop()`, never inside an HTTP transaction.
### The switch, and why `/api/sleep` is its own endpoint
The console home page carries an **On / Sleep** pair in the device card, styled like `/mqtt`'s channel buttons — the console's existing way of picking between states. It is a *view* of the device's state: K1 and MQTT change the same thing, and the page's 3-second `/api/status` poll follows them within a beat.
`POST /api/sleep` only *queues* the transition (`PatternflowSleep::request()`), so its reply reports the state as it stands, not as it will be — stopping a DMA engine and reclocking the CPU belong in `loop()`, not inside an open response. The page therefore paints optimistically and suppresses the poll for 1.5 s, or the reply would snap the switch back under the cursor.
Sleep control deliberately does **not** live on `/api/display`, where the plumbing first landed. That endpoint is independently compile-out-able (`-DPF_DISPLAY_HTTP_ENABLED=0`), which would leave the switch dead in a build with an otherwise complete console — and sleep is a power state, not a display calibration. `/api/display` still *reports* `sleep`, read-only, because a dark panel is the first thing to rule out while tuning.
### MQTT
`<prefix>/sleep` is obeyed in **either** role, unlike the knob and pattern topics — a panel set to Publisher is still a panel you want to be able to switch off. It is a broadcast for the same reason `<prefix>/message` is: one `1` on a shared prefix sleeps every panel on it. Give a panel its own prefix if that is not what you want, and send the command non-retained unless you really do want panels coming back asleep after every reconnect.
The device publishes `<prefix>/sleep/state` (`0` or `1`, non-retained) on every change and once per connection — enough for a Home Assistant switch.
```bash
mosquitto_pub -h <broker> -t patternflow/sleep -m 1
mosquitto_sub -h <broker> -t 'patternflow/sleep/state'
```
Unrecognised payloads are ignored rather than guessed at; a typo should not black out a panel.
### Compile-time flags (`config.h`)
| Flag | Default | Meaning |
| :--- | :--- | :--- |
| `PF_SLEEP_ENABLED` | `1` | compile the feature in at all |
| `PF_SLEEP_STOP_DMA` | `1` | stop the HUB75 transfer, not just blank. **Set to 0 if the panel comes back black or garbled after waking** — sleep then blanks only, which is safe on any panel and still saves the LED current |
| `PF_SLEEP_CPU_MHZ` | `80` | clock while asleep; `0` disables the switch |
| `PF_AWAKE_CPU_MHZ` | `240` | clock restored on wake |
`PF_SLEEP_STOP_DMA` depends on `resumeDMAoutput()`, a Patternflow addition to the vendored HUB75 driver — upstream's `stopDMAoutput()` is a one-way trip. See `src/hub75/VENDORED.md`.
## Remembering the selected pattern
The pattern you left running comes back after a sleep and after a full power cut. It is stored in NVS as a **slug**, not an index: installing or deleting one `.pfm` renumbers the whole list, so an index would come back as somebody else's pattern. Resolution on boot goes through `findPatternByName()` in `pattern_registry.h`, which matches display name and slug alike.
The write is debounced — it waits until SELECT mode is left and the choice has been still for 3 seconds — so spinning K4 through fifty patterns is one NVS write, not fifty. Brightness has used the same shape for longer.
A remembered pattern that is gone, or that no longer loads, falls back to Origin with a log line. The PATTERN FAILED screen is the right answer for a pattern somebody just picked and the wrong one for a boot: nobody asked for it in this session.
## Audio-react, OSC, MIDI, MQTT
None of these is in the core. They are features, and each ships in an edition you install from [the shelf](https://patternflow.work/editions): audio-react, OSC and network MIDI in **Audio**, MQTT in **Performance**. What a client sends is a written contract in `docs/` — [`audio-ws-spec.md`](../docs/audio-ws-spec.md), [`osc-spec.md`](../docs/osc-spec.md), [`midi-spec.md`](../docs/midi-spec.md), [`mqtt-spec.md`](../docs/mqtt-spec.md) — and the guides are [`AUDIO_GUIDE.md`](../AUDIO_GUIDE.md) (sound in, MIDI, OSC) and [`docs/midi-ableton.md`](../docs/midi-ableton.md). The ready-made clients are the audio extension and phone app in [`tools/`](../tools/README.md) and the Max for Live bridge in [`integrations/ableton/`](../integrations/ableton/).
From a pattern's point of view every one of them is the user turning the four encoders: audio bands, OSC deltas and MIDI CCs arrive as `knobDeltas` (or pin a lane on the absolute parameter bus), so an encoder-driven pattern reacts with no transport-specific code.
Two switches, for a local build:
- **Compile-time** — whether a feature is linked in at all is the bundle: `firmware/bundles/<edition>/features_local.h` lists the features an edition carries ([`bundles/README.md`](bundles/README.md)). A feature's own tunables live in `features/<name>/<name>_config.h` (the audio WebSocket port and knob scale in `features/audio/audio_config.h`, the OSC ports in `features/osc/osc_config.h`, the broker defaults in `features/mqtt/mqtt_config.h`), every one `#ifndef`-guarded so a line in `patternflow_secrets.h` wins. `PF_OSC_ENABLED 0` in the secrets file compiles OSC out of an edition that would otherwise carry it; the K2 info screen then shows `OFF (compile-time)`.
- **Runtime** — once compiled in, OSC and MIDI are switched on and off from the panel's NETWORK screen (hold K2), the state saved in NVS; the K2 screen shows `WAIT HOST` until the first OSC packet teaches the panel your computer's address.
## Wireless update from the browser
Once the device is on Wi-Fi, it can flash itself from any browser on the same network — no Arduino IDE, no USB cable, no TLS setup ([#232](https://github.com/engmung/Patternflow/issues/232)).
1. Get a firmware `.bin`: build one on [patternflow.work](https://patternflow.work) and hit *Download .bin*, or export one locally (`arduino-cli compile --output-dir …` — use the app image, not a merged full-flash image).
2. Open `http://patternflow.local` — the device serves a small console (audio sync / firmware update) — and pick **Firmware Update**, or go straight to `/update`. If `.local` doesn't resolve (common on Android), use `http://<ip>/update`; the device shows its IP on the NETWORK screen (hold K2) and on the UPDATE screen.
3. Drop the `.bin` on the page. The panel shows flash progress; the device verifies, reboots, and comes back on the new firmware in about ten seconds.
Notes:
- **Uploads are accepted at any time by default** (`PF_WEBUPDATE_ALWAYS_ARMED 1`): drop a .bin whenever, no trip to the device. The flip side, stated plainly: anyone on the same Wi-Fi can flash the device from a phone browser — the same exposure ArduinoOTA's no-password default already has. On shared, office, or exhibition Wi-Fi, build with `#define PF_WEBUPDATE_ALWAYS_ARMED 0` in `patternflow_secrets.h`: uploads are then refused (`403`) unless the UPDATE screen is open on the device (hold **K2** → NETWORK, turn **K4**) — a physical arming switch only someone at the device can flip; leaving the screen (K4 click, or the 10-minute idle timeout) disarms it again.
- A failed or interrupted upload leaves the old firmware running — `Update.h` only switches the boot partition after a complete, verified image. Power loss *during* the flash write is the one case to avoid; the panel says so while flashing.
- Like all wireless paths, this capability has to arrive over USB once (it ships in the stock release).
- Set `#define PF_WEBUPDATE_ENABLED 0` in `patternflow_secrets.h` to compile it out.
## OTA Updates (For Developers)
The firmware includes `ArduinoOTA` for wireless flashing from the Arduino IDE — no USB cable, no port juggling.
### One-time setup
1. Copy `patternflow/patternflow_secrets.example.h` to `patternflow/patternflow_secrets.h` and fill in your local Wi-Fi credentials:
```cpp
#define PF_WIFI_SSID "your-wifi-name"
#define PF_WIFI_PASS "your-wifi-password"
```
(You don't need to enable `PF_OSC_ENABLED` — OTA brings up Wi-Fi on its own. One secrets file now covers Wi-Fi, OTA, OSC, and audio-react.)
2. Flash once over USB as normal. On boot, the serial console should print:
```
[OTA] Ready — hostname "patternflow.local", IP 192.168.x.x
```
### Subsequent uploads
1. In Arduino IDE, open **Tools → Port** — you should see `patternflow at 192.168.x.x (ESP32)` alongside the USB ports.
2. Select that network port and hit Upload. The IDE compiles, pushes over Wi-Fi, and the device reboots into the new firmware.
3. Progress prints to serial as `[OTA] 47%` etc.
If the network port doesn't appear, make sure your computer and the device are on the same Wi-Fi subnet, and that no firewall is blocking mDNS (UDP port 5353) or the OTA port (3232).
### The upload-password prompt (and how to avoid it)
The firmware ships with **no OTA password** (`PF_OTA_PASSWORD ""`), so the device never challenges for one. But **Arduino IDE 2.x always pops a password dialog for network ports and refuses an empty field** — that's an IDE limitation, not the firmware. Two ways around it:
- **Stay in the IDE:** type any dummy character in the prompt. With no-auth firmware the device ignores it, so the upload still succeeds.
- **Skip the prompt entirely (recommended for no-password):** upload from the command line, which never asks. With the ESP32 core's bundled `espota.py`:
```bash
python espota.py -i patternflow.local -p 3232 -f /path/to/patternflow.ino.bin
```
(no `-a` flag = no password). Or `arduino-cli upload -p patternflow.local -b esp32:esp32:<board> patternflow/`.
To require a password instead, set `#define PF_OTA_PASSWORD "your-secret"` in `patternflow_secrets.h`.
### Known issue: `invalid int value: '{upload.port.properties.port}'`
Arduino IDE 2.x's mDNS discovery for ESP32 core 3.3.8 doesn't populate the `{upload.port.properties.port}` placeholder, so espota receives the literal string and fails:
```
espota.exe: error: argument -p/--port: invalid int value: '{upload.port.properties.port}'
```
One-time fix — create a `platform.local.txt` next to the ESP32 core's `platform.txt` (the IDE auto-merges local overrides, so this survives ESP32 core updates):
- Path on Windows: `%LOCALAPPDATA%\Arduino15\packages\esp32\hardware\esp32\3.3.8\platform.local.txt`
- Path on macOS: `~/Library/Arduino15/packages/esp32/hardware/esp32/3.3.8/platform.local.txt`
- Path on Linux: `~/.arduino15/packages/esp32/hardware/esp32/3.3.8/platform.local.txt`
Contents (one line):
```
tools.esp_ota.upload.pattern={cmd} -i {upload.port.address} -p 3232 "--auth={upload.field.password}" -f "{build.path}/{build.project_name}.bin"
```
This hardcodes the OTA port to 3232 (which is what ArduinoOTA always listens on anyway). Restart the Arduino IDE after creating the file.
### Disabling / customizing
- Set `#define PF_OTA_ENABLED 0` in `patternflow_secrets.h` to compile OTA out entirely (no Wi-Fi stack pulled in unless OSC or audio-react is also enabled).
- Set `#define PF_OTA_HOSTNAME "yourname"` to advertise as `yourname.local` instead of `patternflow.local` — useful if multiple devices are on the same network.
- OTA ships with no password by default (`PF_OTA_PASSWORD ""`). Set `#define PF_OTA_PASSWORD "your-secret"` in `patternflow_secrets.h` to require one on a shared network. See "The upload-password prompt" above for the Arduino IDE 2.x quirk.
## License
MIT - see root `LICENSE-MIT` file.
ea39f3ad2This file is pinned to the selected release. Download the original to open it in your design tool.