◢ The proposition A build matters before it peaks. A consonant chord can remain unstable because it sits far from tonic. The same motif carries different information when it returns. Psychodeli turns those time-dependent musical conditions into signals a visual system can compose with.
Each of those conditions is invisible inside a single audio frame. Level, brightness, and spectral shape describe the present moment. A build, a distance from home, a return: none of them exists in less than a few seconds. Read frame by frame, the signal gives up the drop but not the approach to it. A chord can be loud without being far from home. A repeat is not yet a return. The models below keep that history.
When it works you can see it without reading a meter. The picture follows the shape of the music instead of its level: it leans into a build, and holds through a lull. A hook that comes back is not drawn the way it was drawn the first time. The panels below expose the signals under those moves, so the claim can be checked.
“Cognitive” names the level of the model, not a claim about consciousness or human equivalence. The current engine computes music-informed proxies for listener-relevant structure. These tests expose what it measures, what it infers, how the picture uses the result, and where that chain still breaks. Three rules hold for every section below. Every number is the engine's own reading, and none has been checked against human annotation. Citations mark design lineage, not validation. A detector that is wrong, silent, or wired to nothing gets reported here.
This page is a runnable bench. Each panel synthesizes a controlled sound and sends it through the same analysis path used by the desktop app. The meters expose audio features, model proxies, and rendered parameters beside the picture. They demonstrate current engine behavior. The ledger later shows the same channels on complete tracks as provenance and diagnostics.
§1 · Expectation Live
The build does its work before the drop arrives.
This test isolates one narrow behavior: a model signal can begin changing before raw energy reaches its maximum. Psychodeli combines the rate of change in energy, spectral brightness, onset density, and drum momentum into a slow feature accumulator. The product calls that channel arousal; here it is a build-up proxy, not a measurement of anyone's physiological state.
The accumulator rises through the approach and releases more slowly after the cut. That lead and lag give the visual system a longer gesture than an amplitude follower can provide. The design draws on theories that separate anticipation from an event's arrival.
Press play. The top two meters follow immediate audio level. The build-up proxy should climb across the riser and remain elevated after the sound stops. That timing difference is the working claim in this panel.
A planned depth gesture is also visible in the console because its failure matters: the engine arms the tunnel target, but the rendered parameter remains near zero in this probe and in the current full-track regression reel. The full plunge has appeared only under a forced test. tunnel · wired, not yet opening
The build real engine
The top two meters follow the sound. The build-up proxy integrates their change and releases more slowly. Depth is the value received by the shader; it currently remains near zero even when the engine arms the intended gesture.
No microphone. This page synthesizes the sound and feeds it through the desktop app's system-audio analysis path.
How the arousal source works
A fast trace follows immediate energy while a slower reservoir integrates the rate of rise of loudness, spectral brightness, onset density, and drum momentum. It attacks quickly enough to catch a build and releases slowly enough to dwell after it. When loudness and accumulated tension peak together, the response grows more than either input could produce alone. The resulting proxy joins other model signals through an additive composition layer, so several kinds of musical evidence can shape one visual gesture without taking turns at the controls. Meyer (1956), Huron (2006), and Salimpoor et al. (2011) provide design lineage for separating anticipation from arrival; they do not validate this accumulator.
How the anticipation tunnel works
The build-up reservoir is continuous; the tunnel is a decision. When a section-change detector
crosses its trigger, the engine begins an extended depth gesture toward the predicted event. The
direction is deterministic because anticipation must aim somewhere: a random plunge would read as noise.
The design target reaches roughly −0.9 of depth at the peak, then releases halfway on arrival.
The sense of travel combines a corkscrew flow with ring-waves moving down the throat. The vanishing point dims and loses saturation, so depth reads as distance rather than a simple change of scale. On release, the flow completes its phase before settling.
What broke in testing. Foresight arms the sweep on anticipatory triggers, but the channel doesn't yet open. A section-scale build→drop reel pushed arousal to its ceiling and fired foresight mid-build, yet the depth drifted slightly the wrong way and never went negative. The arming and the sweep activation appear desynchronized, so the full plunge has only appeared under a forced probe. That reel now serves as the regression test for the fix. The intended timing follows the imagination and tension responses that precede an event (Salimpoor et al. 2011; Huron 2006).
§2 · Tonal tension Live
A clean chord can still be a long way from home.
The model first estimates tonal home by comparing the recent balance of pitch classes with established profiles for major and minor keys. It then keeps two readings separate: surface dissonance, the immediate friction among sounding intervals, and tonal distance, the current harmony's position relative to the estimated tonic.
A chord can score low on surface dissonance while sitting far from home. Psychodeli blends the two readings into a visual tension channel. That equal blend is a design heuristic informed by tonal-space research, not a measurement of a listener's felt tension.
Press play. A C-major scale primes the key estimate, then the synth loops I → IV → ii → V7 → I. The panel exposes the two inputs and their blend by chord. The V7 should raise surface friction and distance; resolution to I should lower both. This test establishes the engine's response to a controlled cadence; listener validation remains future work.
Tension & resolution real engine
The visual control signal is an equal blend of the two above. The V7 peaks both; a tritone cluster peaks surface; a remote-but-clean chord peaks distance. Resolution to I drops all three.
Key and chord labels come from the live estimator. Relative major/minor ambiguity remains; because this probe deliberately primes C major, an A-minor reading here is an error rather than an equally valid interpretation.
How the two kinds of tension work
First the model estimates tonal home by comparing the recent distribution of pitches with the stability patterns listeners report for major and minor keys. Once home is credible, it measures how far the current harmony has travelled from it: tonic, fifth, and third sit deep in the hierarchy; other diatonic notes sit nearer the surface; chromatic notes live at the edge. That tonal distance is combined with surface dissonance, the immediate friction of intervals such as minor seconds and tritones. A chord can therefore be smooth but far from home, or locally abrasive while remaining in the key. When no stable key emerges, tonal distance abstains and surface dissonance carries the reading alone, so percussive and atonal material do not acquire a fictional tonic (Lerdahl & Krumhansl 2007).
§3 · Recognition Live
When the hook returns, the picture can remember it.
A repeated phrase gives a visual system information that does not exist in the current audio frame: this material has happened before. Psychodeli keeps acoustic fingerprints from the recent past and compares them with the incoming stream.
When a similarity holds for several seconds and stands out from its surroundings, the engine stores the fingerprint as a recurrence candidate. A later match can ask the director to recall a related visual state. On complete sessions, a longer store applies the same mechanism to returns separated by minutes.
Press play. The synth states a motif, pauses, then repeats the exact audio. The first pass should create a stored candidate; the second should raise the recurrence score. This probe tests exact acoustic return. It does not yet test recognition across transposition, tempo, or timbre, and a score that rises after the repeat begins is not evidence of anticipation.
Motif recognition real engine
The engine scans a diagonal similarity matrix of the recent past. The first play establishes the fingerprint candidate. The second can produce a recurrence hit.
This score can request a coordinated visual reprise. The current probe establishes an exact acoustic repeat, not a cognitively meaningful motif or structural interpretation.
How a motif becomes a memory
The engine compares each live acoustic fingerprint with roughly the previous 200 seconds. A brief match is not enough: the resemblance must hold for several seconds and stand out from its surroundings before it becomes a stored recurrence candidate. When the same entry cue returns, the engine reports how strongly the live passage aligns with that fingerprint. Duration and contrast keep a stray repeated hit from masquerading as a phrase.
Two shelves of memory — short-term chunks, long-term rhymes
The short-term store compares fingerprints across roughly the previous 200 seconds. Beneath it, a long-term store spanning about fifteen minutes and a high-score exact-recall tier keep fingerprints and past visual states on hand, so a callback can land long after the short-term memory has rolled over. Each recall arrives through the same brief ease as a short-term return, so a rhyme from fifteen minutes ago eases into the current picture instead of cutting into it. A returning chorus gains force from the memory of the first one (Huron 2006; Margulis, On Repeat, 2014).
§4 · Coherence Live
One kick drum should not knock a sixteen-bar gesture off course.
Dozens of systems write to the same few visual parameters. A beat accent can arrive while a section-length sweep is still moving. If the latest writer always wins, the sweep gets kicked off its line every few hundred milliseconds and the picture stutters.
Psychodeli ranks each request by musical timescale. The section sweep keeps control while the beats add detail around it. A velocity-and-jerk limiter rounds abrupt changes, and a render-rate glide fills the gaps between the ~20 Hz analysis ticks.
How long gestures survive short accents
Every visual proposal carries a timescale. A section-length sweep outranks a frame-length beat accent, so the larger phrase keeps temporary ownership while the kicks decorate it. Once a value is committed, one smoothing chain limits velocity and jerk — the rate at which acceleration changes — so sharp decisions still arrive with rounded motion. A render-rate glide fills the space between the slower analysis updates, and the whole system is rate-independent: it behaves consistently at 30 fps on a phone and 120 Hz on a desktop. Continuity and common fate let the visual system read a moving field as one thing instead of many (Wertheimer 1923; Johansson 1973).
§5 · The ledger Live
We started recording because the live meters kept forgetting.
Ten times a second, the desktop app records 85 signals: audio measurements (energy, bands, per-stem presence), model proxies (build-up, tonal tension, melodic arc), and the shader uniforms the picture received — while a second lane logs the controller's decisions as they happen: phase changes, pattern replays, theme returns, every prediction it makes. One file per song, keyed by content identity rather than filename (loopback audio has no filename), compressed and indexed. The ledger currently holds 500 tracks across 2,223 plays. That is an observation archive, not a validation dataset.
Below is one page of it: a four-minute electronic track played through the desktop app; three model channels drawn over the controller's event lane. The build-up proxy rides above the raw energy it integrates. The tension heuristic changes with the harmony. At 2:38, the recurrence system matched material from 2:04 earlier and requested a visual reprise with an internal score of 0.87. That score has not been calibrated against human annotations.
The first play is the soundcheck. Offline, every capture gets a second pass: a clean layer with tempo settled to the right octave, retrospective section boundaries, and anticipatable cues. A causal, real-time analyzer can never have those; offline, you're allowed to look forward across the whole signal. When the same song returns, the engine loads that clean layer and can ask "what's coming in the next six seconds?" — lookahead instead of prediction. The query surface is live in the app today; wiring it into the director's anticipatory moves (§1's tunnel, for one) is the work in progress. lookahead consumers · wiring
What one row of the ledger remembers
Ten times a second, one row records audio measurements, model outputs, and shader parameters. A parallel lane records discrete events such as section changes, predictions, and visual returns. Rows are batched away from the render loop, then compressed into one artifact keyed by song identity rather than filename — a necessity when loopback audio has no file to inspect. When the reported song changes, one artifact closes and the next begins. Long live sets rotate periodically so a single ambiguous title cannot swallow an evening.
The offline half — from capture to reference analysis
The clean layer is baked in two tiers. Tier A runs when capture closes: octave-settled global tempo and a corrected beat grid. Tier B adds whole-song structure — separated musical parts, section boundaries, and downbeats — then merges those results back into the same artifact. Local files can enter at this second tier without a prior listening pass. On top of that sits the recurrence work: a repeated musical span is only admitted as a family when the pitch lane and the rhythm lane independently agree on the same beat-grid span — a discipline that cut the review surface from 3.5–10 candidate repeats per minute to at most one root family per track. That analysis is an internal reference for grading the live recurrence ladder (§3). It is not ground truth until independently annotated and evaluated.
Lookahead replaces prediction
The engine's old predictive layer once reported ~99% confidence at ~16% accuracy. Bayesian recalibration made the guess honest — it did not make it right. Lookahead sidesteps the problem: on replay of a recognized song, the clean layer can supply real future beats, sections, and cues in a six-second window instead of pretending to forecast them. The query surface works today; feeding it into the anticipation tunnel, beat preparation, and the director's foresight remains the next phase. The clean layer stays a source, not a writer: it informs the director; it never bypasses the parameter arbiter (§4).
Beyond the waveform
A waveform shows changes in amplitude. A spectrum shows where the energy sits. Those measurements are the starting point for a richer kind of audio reactivity: identifying what is sounding, following how the music unfolds, and composing a visual response across phrases and sections. Apple's SoundAnalysis makes one part of that advance available as a built-in platform capability: a classifier for over 300 sound categories, including musical instruments and human vocal sounds. A visual system can respond to the arrival of a voice or a string texture, rather than treating every equally loud sound as the same event.
Psychodeli combines instrument-family cues with its own models of rhythm, harmonic tension, melodic motion and recurrence. The conductor coordinates their requests so a beat accent can coexist with a longer gesture. The aim is for a returning phrase to recall a look, an instrumental entrance to reshape a scene, and a quiet passage to leave space. Audio reactivity becomes musical direction: the picture gains context, memory and pacing, instead of starting over with every audio frame.
Where the Apple analysis stands today
As of October 3, 2026, our local-file reference covers 5,680 tracks and about 369 hours of analyzed audio. Agreement with Apple is useful evidence, but its labels still need independent listener checks. A blind listening pilot pairs 12 candidate changes with 12 shifted controls from the same recordings: do people actually hear the claimed entrance or exit? Results are still pending. In six excerpts played through MacBook speakers into its microphone, broad family identity often survived, but confidence and timing changed substantially. File analysis alone cannot certify what a room microphone hears.
§6 · Live Score Transport next
The next cable carries the ledger into TouchDesigner.
A captured analysis session contains section boundaries, estimated section identity, detector scores, revisions, source generations, and the model traces in the ledger above. Live Score turns that record into a portable semantic stream for TouchDesigner and, later, any creative system that speaks OSC.
The first public vocabulary is deliberately narrow. A receiver learns that a boundary committed, what kind of section followed, whether that identity was revised, and the detector score attached to it. It gets stable IDs and sequence numbers so a receiver can recover from UDP loss. TouchDesigner can use those events to change a scene, protect a vacancy, or remember a return without rebuilding Psychodeli's analysis network in CHOPs.
Capture the analysis
Live The desktop app records 85 signals at 10 Hz plus the controller's event lane, then closes a per-track artifact with source identity and provenance.
Project committed analysis
Core verified The transport-neutral projector emits fixed OSC-shaped
messages for section.boundary and section.identity, including revisions,
confidence, freshness, and recovery state.
Patch the show
Next Bounded Electron UDP, destination health, and a reference TouchDesigner receiver turn the verified message contract into a cable another visual system can use.
→ /psychodeli/v1/event/section/identity/commit section:7 · chorus
↻ /psychodeli/v1/event/section/identity/relabel chorus → bridge · revision 1
Psychodeli handles the musical analysis. TouchDesigner keeps control of the visual system. Live Score is the proposed cable between them.
§7 · The director Live
In one 3:42 track, the director changed phase nine times.
The Algorithmic Exploration controller maps model signals to visual policies. It runs a four-phase creative cycle — discovery (try new combinations), development (refine what's working), variation (bend the established), return (call back to the memorable) — over two stores of recent and session-scale state plus a growing book of audio→visual fingerprints. When a fingerprint crosses its match threshold, it replays the associated visual move — a turbo replay.
Read the capture in §5 again, top band this time. The log records why each phase
changed: to variation at every chorus, to
development on the verses, back to return at both breakdowns. It fired 23 turbo replays,
matched 18 long-term patterns, and landed the two theme returns in the bottom lane. None of that
was scripted in advance. The event labels (section:chorus, cue:breakdown) came from
live detectors; the policy selected a phase in response.
A purpose-built test can ask a sharper question: when the composer knows a phrase has returned, which of the director's repeat systems agrees?
Explorer-max stress pass · captured 2026-08-27
The return stack disagrees with itself.
The score states A, leaves it, then returns the same source buffers. It also offers a stripped dub vacancy, a five-semitone transposition, and a half-time impostor. The Play authors no visual move: its eight cues are truth markers in the log, not instructions to the picture.
- A0literal
- Bcontrast
- A1literal return
- Dstrip
- A+5transpose
- Cimpostor
- A2/A′return + evolve
Widths are proportional to bars: 4 · 4 · 4 · 2 · 4 · 2 · 4.
That disagreement is the result. Turbo is active and highly gated; 569 candidate actions were suppressed. The chunk system compiled repeat-shaped material but never called a later entry a return. The span detector abstained even with its documented k-nearest-neighbour experiment enabled. Offline analysis found 24 candidate families but accepted none while held-out calibration is still pending. This pass demonstrates the recognizers' boundaries; it does not claim they identified the authored A returns.
Open the machine-readable evidence →The same log attaches internal verdicts to predictions. On this track it recorded 2,444 predictions, 2,321 scored, and 62% marked landed. Those figures are diagnostic, not a benchmark: the page does not supply task definitions, tolerances, baselines, or class balance. The useful failure is specific. Beat timing and vocal-onset events scored highly under the internal rules, while chord-stability and phrase-arc went 0-for-547. Those zeros are why §5's lookahead work exists.
What the director remembers, and what it writes down
Short-term memory holds the recent musical and visual context; a long-term store keeps session-scale patterns for roughly fifteen minutes with match-score-weighted recall, so a callback lands as a composed arrival (eased ~800 ms), not a cut. The ledger records each creative-phase change and its reason, every reflex replay, each long-memory match, and the lag and detector score behind a theme return. That is how the 2:38 return in the figure can be traced to material measured 2:04 earlier with an internal score of 0.87.
The full spine
Here is what works, what is half-wired, and what still exists on paper.
The badges carry the status: Live works today, In progress is partly wired, Next is still design on paper.