Overview

Bring Your Character to Life.

CosNFX brings cosplay, LARP, and stage-magic characters to life: natural arm movements — a punch, a wrist flick, an overhead raise — fire high-fidelity sound effects and elemental energy bursts straight from the wrist. It runs as a standalone Wear OS app (no phone required to perform), tested on a Galaxy Watch 7.

Each character is an identity with its own colour, sound kit, energy field, and cinematic backdrop — 30 built-ins (heroes, monsters, and mages: Superhero, Cyborg, Pyromancer, Vampire, Reaper, Werewolf, Robot, and more) plus your own. From a phone browser you assign gestures, upload your own .wav/.mp3, chain gestures into combos, and route audio — with nothing to install on the phone. The phone companion app mirrors that whole control surface natively over the Bluetooth data layer, so a venue with no Wi-Fi changes nothing.

Active beta   Kotlin + Jetpack Compose (Wear Material 3) · built across a watch app and a phone companion · com.jdcap.cosnfx v2.10.0. Free to install, with an optional one-time Pro in-app purchase.

At A Glance

Watch Core, Phone Mission Control

The watch is the impact core; the phone is the configuration surface and an optional second speaker. A single Android foreground service (HeroService) owns the audio engine, the sensors, and an embedded web server, so they survive the screen sleeping or the app backgrounding.

Watch

One Always-On Service

A single background service (HeroService) hosts the self-healing AudioTrack engine, reads the watch's motion sensors, and detects your gestures. The watch UI is one page — a Clean HUD: the identity name is the arm toggle, ringed by numbered 1–7 gesture nodes over a cinematic per-identity backdrop.

Phone

Mission Control in Your Browser

The watch runs an embedded Ktor server on :8080; any phone browser on the same Wi-Fi opens an 8-tab panel — Profile, Gestures, Looks, Combat, Library, Props, Calibrate, Settings — over REST + a WebSocket event stream, gated by a 6-digit pairing PIN shown on the watch. Nothing to install on the phone — and the companion app mirrors the same tabs natively over Bluetooth, and can only reach an approved set of commands.

Visuals

Per-Identity Energy Fields

Each identity renders a transparent particle energy field over its cinematic backdrop — a native Kotlin port of the web visualization engine (dozens of built-in effects) running on a render-thread SurfaceView with additive bloom, frame-throttled so it pauses when the screen's off. (The old AGSL GPU shader and the earlier Compose-Canvas effects are retired.)

Audio Engineering

Low-Latency by Design.

Standard Android media players add 40–100 ms of round-trip latency — unacceptable for a sound triggered by a physical wrist flick. CosNFX bypasses them and streams raw 16-bit PCM straight into a hardware AudioTrack on a MAX_PRIORITY thread, requesting PERFORMANCE_MODE_LOW_LATENCY (the fast HAL path, with a fallback build for devices that reject it).

The engine is self-healing: build, play, and render are wrapped in a retry loop that releases and rebuilds the track after a Bluetooth/audio-focus change (which used to throw uncaught and kill the process). Writes use 256-sample chunks (~6 ms — latency negligible, far fewer underruns). To save power the track pauses when idle, but that pause is arm-gated: it stays warm (~2.5 s) while armed so every gesture in a performance fires instantly, and pauses fast (250 ms) when disarmed.

Sensor Fusion

Measured-Axis Gesture Detection

The hard problem is firing on a real gesture and never on natural conversation. The honest fix was to stop trusting the textbook: on this Galaxy Watch 7 the sensor axes aren't what the docs assume, so each gesture is keyed to the axis it actually drives — read from a live on-watch inspector.

Measured axes

One Axis Per Gesture

Gravity X is the arm up/down axis (gravity Y stays ~0 in every pose), gating the armed state and the overhead Raise. The two wrist flicks (Underhand / Overhand) read gyro X; the thrust and impulse moves — a forward Strike, a superhero-landing down-punch, a hip thrust, and a Fist clench — each key off the accel axis they actually drive, picked by which signal dominates so they can't trip each other. (The fist clench replaced a finger-snap detector after on-watch measurement proved a real snap's impulse sits below ordinary arm movement at the 100 Hz sensor ceiling — the data said no, so the gesture changed.)

Anti-cross-talk

Reorientation Suppression

A fast vertical sweep of the arm (Δgravity X > 1.0 — i.e. you're raising it) mutes the twitchy accel detectors for 400 ms, so the swing itself can't trip a thrust. All lanes share a 350 ms global cooldown — one gesture per motion — and a hands-free double-tap arms or disarms above all of it.

Calibration

Teach the Watch Your Motion.

Fixed axes fit how the prop sits on this wrist and how this performer moves; a very different pose can drift. So calibration is built in: record a built-in impulse gesture a few times and the watch auto-aims that gesture's detector at the channel your motion actually drives — fitting detection to you instead of a hard-coded axis.

It's per-gesture (Calibrate / Redo), with Recalibrate-all and Reset-to-defaults, and it's an opt-in backup, never a regression: an uncalibrated gesture keeps its built-in detection unchanged, and an under-sampled capture is rejected rather than saved.

Props & Output

From the Wrist to the Room

A gesture doesn't have to stop at sound — it can also fire an external action and be routed across devices.

Props

External Lights & Devices

An opt-in per-gesture action fires alongside the sound. An HTTP call drives anything with a local web API — WLED strips, Philips Hue, Home Assistant (smoke machines, fans), DIY ESP32/Arduino — while OSC and DMX (Art-Net) reach lighting consoles and show-control rigs, so one move can flash a costume and cue a fog burst together. Each row has a Test button, and HTTP targets are scoped in code to private LAN addresses (anything public is forced to HTTPS).

Audio routing

Multi-Device Audio & Battle Mode

A main/support mix routes independently across three outputs — the watch, the phone companion app, and the Mission Control browser tab — so the main hit lands on a loud phone speaker while a support layer stays on the wrist (any mix, or silence either), and a gesture can play across several devices at once. An opt-in Battle mode links two or more watches over Wi-Fi for versus play — a strike deals damage, a held Raise blocks, and health regenerates while disarmed.

Engineering Reality: Battery vs. Latency

A performance prop has to stay instant without draining the watch mid-show. Sensors run full-rate only while armed and downshift when disarmed; the audio track stays warm while armed and pauses fast when not. The screen (kept on while armed) is the real drain — so the watch auto-disarms after ~15 minutes idle (any gesture resets it), letting the screen sleep and the watch fall to its idle floor. CosNFX is in active beta development.