Overview

Bring Your Character to Life.

CosNFX brings cosplay, LARP, and stage-magic characters to life: natural arm movements — a punch, a wrist flick, an overhead raise — fire high-fidelity sound effects and elemental energy bursts straight from the wrist. It runs as a standalone Wear OS app (no phone required to perform), tested on a Galaxy Watch 7.

Each character is an identity with its own colour, sound kit, and energy field; swipe to switch between built-ins (Superhero, Cyborg, Sorcerer, Warrior, Netrunner, and Halloween Ghost / Reaper / Witch) or build your own. From a phone browser you assign gestures, upload your own .wav/.mp3, chain gestures into combos, and route audio — with no app to install on the phone.

Active beta   Kotlin + Jetpack Compose (Wear Material 3) · ~4,400 lines of Kotlin across 9 modules · com.jdcap.cosnfx v2.5.2.

At A Glance

Watch Core, Phone Mission Control

The watch is the impact core; the phone is the configuration surface and an optional second speaker. A single Android foreground service (HeroService) owns the audio engine, the sensors, and an embedded web server, so they survive the screen sleeping or the app backgrounding.

Watch

HeroService + HeroSynth

A foreground service hosts the self-healing AudioTrack engine, gravity/accel/gyro sensor fusion, and the measured-axis gesture detectors. The watch UI is one page: a split sun/moon core ringed by per-gesture elemental orbs.

Phone

Mission Control (8 tabs)

The watch runs an embedded Ktor server on :8080; any phone browser on the same Wi-Fi opens an 8-tab panel — Gestures, Combos, Look, Combat, Library, Props, Calibrate, System — over REST + a WebSocket event stream. Nothing to install on the phone.

Visuals

AGSL GPU Energy Fields

Per-identity energy fields render on a GPU AGSL shader (with a Canvas fallback), plus audio-reactive elemental bursts per gesture — frame-throttled so they pause when the screen is off.

Audio Engineering

Low-Latency by Design.

Standard Android media players add 40–100 ms of round-trip latency — unacceptable for a sound triggered by a physical wrist snap. CosNFX bypasses them and streams raw 16-bit PCM straight into a hardware AudioTrack on a MAX_PRIORITY thread, requesting PERFORMANCE_MODE_LOW_LATENCY (the fast HAL path, with a fallback build for devices that reject it).

The engine is self-healing: build, play, and render are wrapped in a retry loop that releases and rebuilds the track after a Bluetooth/audio-focus change (which used to throw uncaught and kill the process). Writes use 256-sample chunks (~6 ms — latency negligible, far fewer underruns). To save power the track pauses when idle, but that pause is arm-gated: it stays warm (~2.5 s) while armed so every gesture in a performance fires instantly, and pauses fast (250 ms) when disarmed.

Sensor Fusion

Measured-Axis Gesture Detection

The hard problem is firing on a real gesture and never on natural conversation. The honest fix was to stop trusting the textbook: on this Galaxy Watch 7 the sensor axes aren't what the docs assume, so each gesture is keyed to the axis it actually drives — read from a live on-watch inspector.

Measured axes

One Axis Per Gesture

Gravity X is the arm up/down axis (gravity Y stays ~0 in every pose), gating the armed state and the overhead Raise. Wrist snaps (Underhand / Overhand / Come-here) read gyro X; the three thrust/translation moves each own a distinct accel axis — Hadouken = X, Wave = Y, Punch = Z — detected by which axis dominates, so they can't trip each other.

Anti-cross-talk

Reorientation Suppression

A fast change in the gravity vector (Δgravity > 1.6 — i.e. you're raising your arm) mutes the accel and come-here detectors for 400 ms, so raising your arm can't trip a thrust. All lanes share a 350 ms global cooldown: one gesture per motion.

Calibration

Teach the Watch Your Motion.

Fixed axes fit how the prop sits on this wrist and how this performer moves; a very different pose can drift. So calibration is built in: record a built-in impulse gesture a few times and the watch auto-aims that gesture's detector at the channel your motion actually drives — fitting detection to you instead of a hard-coded axis.

It's per-gesture (Calibrate / Redo), with Recalibrate-all and Reset-to-defaults, and it's an opt-in backup, never a regression: an uncalibrated gesture keeps its built-in detection unchanged, and an under-sampled capture is rejected rather than saved.

Props & Output

From the Wrist to the Room

A gesture doesn't have to stop at sound — it can also fire an external action and be routed across devices.

Props

External Lights & Devices

An opt-in per-gesture HTTP action (method + URL + body) fires alongside the sound — driving anything with a local web API: WLED strips, Philips Hue, Home Assistant (smoke machines, fans), DIY ESP32/Arduino, or IFTTT. Each row has a Test button. It's device-agnostic — you supply the URL — and scoped to the trusted LAN.

Audio routing

Per-Device Mix & Combat Link

A watch/phone × main/support matrix sends the main sound to the phone speaker and the support layer to the watch — any mix, or silence either. An opt-in Combat Link relays strike / shield / damage events between paired devices for co-op or versus play.

Engineering Reality: Battery vs. Latency

A performance prop has to stay instant without draining the watch mid-show. Sensors run full-rate only while armed and downshift when disarmed; the audio track stays warm while armed and pauses fast when not. The screen (kept on while armed) is the real drain — so the watch auto-disarms after ~4 minutes idle (any gesture resets it), letting the screen sleep and the watch fall to its idle floor. CosNFX is in active beta development.