This is a side project. Not a service, not on the price list, and I am not offering to score your film. It is a prototype on one machine in the studio. The browser-only half is public as Musiccer in Labs; the model-backed half is not. I am writing it up because building it taught me things about local models that carry over to the work I get paid for.
The itch: an architectural film gets cut, graded and approved, and then it needs thirty to ninety seconds of music. Stock library — pay per project, then spend an afternoon rejecting the same six corporate-uplifting tracks everyone else uses. Composer — right for a big film, overkill for a lobby loop. Which licence covers which use is the recurring annoyance. Exactly the kind of problem I start poking at on a Sunday.
I already run image models on my own hardware (the whole finishing pipeline), as much for confidentiality as for cost. Audio turned out to be the same problem in a different hat: a handful of open models, each with its own Python environment and its own idea of what a file is, and a lot of unglamorous plumbing between them. The plumbing is the job. It was on the image side too.
One Flask server on one port, nine static pages, every model behind a small HTTP API. No build step, no npm, no database. Generated files land in an output/ folder, one directory per job.
- Symbolic orchestration. SymphonyGen — a hierarchical model conditioned on a beat-quantised harmony skeleton, accepted at ISMIR 2026 — writes orchestral MIDI from scratch or from a chord progression. Three released checkpoints, up to four variations per run.
- Text to MIDI. text2midi from the AMAAI Lab turns a written description into a MIDI sketch.
- Continuation and accompaniment. Stanford’s Anticipatory Music Transformer either keeps composing where a MIDI file stops, or treats it as a melody and writes parts around it.
- Audio. ACE-Step 1.5 as a local subprocess: actual audio from a prompt, with declared BPM and key.
- Analysis, the other way. Basic Pitch transcribes audio to MIDI, Demucs splits a track into four stems, Matchering masters a mix against a reference.
- Sound. One soft synth for the whole app: js-synthesizer 1.13.0 with libfluidsynth 2.4.6 in WebAssembly, loading the same GeneralUser GS soundfont the server-side FluidSynth renders with. Sixteen MIDI channels, one stereo output, so per-track level and pan are CC7 and CC10 rather than a gain node that cannot exist.
Two things do not run on the machine: GrooVAE fetches its drum checkpoints from Google’s CDN on first use, and the tool pages load their MIDI-player library from a CDN.
The front page is a sketchpad: chord progression editor, melody lane, drum grid, sixty-two scales, cadence labelling, reharmonisation suggestions, negative-harmony mirroring, a lyrics lane, Web MIDI in and out.
The drum lane is its own page: a Euclidean step editor per voice, the pattern drawn as rings, two buttons that hand the bar to GrooVAE to humanise or re-play it.
Attached to it is the piece nobody needed: a world map. Scales, drum patterns and time signatures pinned to where they come from, and fifty-five traditional songs from forty-six countries and regions, each loadable into the sketchpad. It started as a way to make an unfamiliar scale mean something. I kept it because I liked it.
From there the material goes to a song editor: a canvas timeline with MIDI and audio tracks, clips you can move, trim, split and loop, waveform peaks per clip, metronome, count-in, save, undo. Transport on integer ticks at 480 per quarter note, a 25 ms scheduler filling a 150 ms lookahead — the standard two-clocks arrangement, which is what makes editing during playback free.
The same page has a second view over the same model: a session grid, borrowed wholesale from Ableton rather than reinvented. One playing clip per track, launches quantised to a bar or a beat, scenes as rows, a QWERTY grid across four scenes and eight tracks. Bouncing renders the MIDI tracks server-side with FluidSynth, sums them against the audio clips in an OfflineAudioContext, and drops the mix back into output/.
The one design decision I would defend is the least clever one. Every producer writes a job directory with a small manifest.json: what made it, when, from which parent job, and whatever metadata it knows, like BPM and key. Every consumer accepts a path relative to output/ instead of demanding an upload. That is the entire interoperability layer. Files as IPC. No database, no schema registry.
It buys a lot for nothing. A picker lists recent jobs with their provenance chain, so “stems ← song ← sketch” reads at a glance. Every result gets a “Send to” menu whose targets are one line each in a routing table. A BroadcastChannel lets a result finished in one tab announce itself to the others.
A prototype, and the hand-offs are still the weak point.
The design note that drove the last round opened by listing the damage: five incompatible note formats in circulation, every hand-off one-way out of the sketchpad, three tools that were dead ends — the drum page never touched the server, generated audio never landed in the shared folder, the orchestrator’s variations fed nothing. Those three are closed. What replaced them is not a finished system, it is a convention that everything currently obeys.
The unfinished parts, specifically: the sketchpad still carries its own copy of the timing code. The song editor edits clips, not notes — no piano roll. Audio follows tempo by changing playback rate, which also changes pitch; proper time-stretching is a dependency nobody has needed enough to add. Above fifteen melodic tracks they start sharing a MIDI channel, because one synth has sixteen. A harmony skeleton is silently looped or cut to thirty-two bars. One heavy job at a time, because one GPU and one lock. Background the tab and the browser throttles the scheduler until playback starves.
None of that is a bug list waiting to be cleared. They are ceilings I chose to get something working, each written next to the code that imposes it. It plays, it generates, it round-trips, and it is nowhere near a product.
Nothing from it has gone into a delivered film. It may never. AI-composed music is on the price list, but that comes out of the production pipeline’s music engines, behind the same render facade as everything else I deliver, not out of this workbench. What I got instead: a much better feel for how far local audio models have come, where they still fall over, and how much of “AI tooling” is actually file management. Roughly the answer the image side gave.
If you build things like this on your own evenings, say hello.