How it works

Liner Lyric Studio

A lyric video is a song, some timed text and a picture that moves with the music. Liner builds all three in one page, with no build step and nothing sent to a server. Below are the seven ideas that make it work, each with the few lines of code that carry it.

Interface design
On-device ML
Motion & typography
Canvas animation
Web Audio & MediaRecorder
Privacy by default

From song to video

Loadthe MP3 is opened in the browser, never uploaded
Timepaste lyrics with [0:12] stamps, or tap along
Stagean on-device model reads each section's mood, or pick a scene by hand
Drawevery frame paints sky, scene, particles, visualizer, lyrics
Exportframes and audio are encoded into one MP4 or WebM file

The ideas

01A screening room, not a form
02Lyrics stay plain text
03Time it by tapping along
04A scene is a small, checked object
05A small model reads the mood
06Paint every frame from scratch
07Record what you see, keep the song at home
→Read further

A screening room, not a form

The video is the biggest thing on the page

The page is laid out like a small cinema. A 16:9 stage takes most of the width, and a narrow rail on the right walks through three numbered steps: song, lyrics, scene. The whole interface is one dark theme on purpose, so the preview reads like a screen and its colours are not fighting a bright page around it.

index.htmllines 12–29

/* Layout: a dark "screening room" — the 16:9 stage dominates, a narrow control rail on the right walks through song → lyrics → scene. Deliberately single dark theme so the video reads like a screen. */
:root{
  --bg:#17141f;
  --panel:#201c2b;
  --raised:#2a2538;
  --line:#3a3349;
  --fg:#efe8f5;
  --muted:#a69db8;
  --accent:#f4b183;
  --accent-ink:#2b190d;
  --lilac:#b9a9ee;
  --danger:#ff8c84;
  --font-ui:"Figtree",system-ui,-apple-system,"Segoe UI",sans-serif;
  --font-display:"Instrument Serif","Iowan Old Style",Georgia,serif;
  --font-mono:"Space Mono",ui-monospace,Menlo,monospace;
  color-scheme:dark;
}
html,body{margin:0;background:var(--bg);color:var(--fg)}

The design intent, written down where the next person editing the CSS will see it.
A small set of colour tokens: three surface levels for depth, one warm accent (peach) for primary actions, and lilac only for focus rings.
Three typefaces with three jobs: Figtree for controls, Instrument Serif for headings and the tap-sync prompt, Space Mono for times and the lyrics editor.
Tells the browser this page is dark, so native controls like the range slider and colour pickers render to match.

index.htmllines 38–44

.grid{display:grid;grid-template-columns:minmax(0,1fr);gap:18px;align-items:start}
@media (min-width:1000px){.grid{grid-template-columns:minmax(0,1fr) 380px}}
.stage-col{display:flex;flex-direction:column;gap:10px;min-width:0}
.stage{position:relative;border-radius:10px;overflow:hidden;background:#000;box-shadow:0 20px 60px -20px rgba(0,0,0,.7);outline:1px solid var(--line)}
.stage canvas{display:block;width:100%;height:auto;aspect-ratio:16/9}
.stage:fullscreen{display:flex;align-items:center;justify-content:center;border-radius:0}
.stage:fullscreen canvas{width:auto;height:100%;max-width:100%;object-fit:contain}

On a phone everything stacks in one column, stage first.
From 1000px wide, the stage gets all the flexible space and the control rail is a fixed 380px. minmax(0,1fr) lets the stage shrink instead of overflowing.
The canvas always draws at 1280×720 internally; CSS only scales it, so the preview and the exported video are the same picture.
Full screen letterboxes the stage on black, the way a projector would.

Lyrics stay plain text

Timestamps in the text are the whole data format

There is no timeline editor. You paste lyrics into a textarea, and a line that starts with a time like [0:12] begins a new block on screen; lines under it without a time show together with it. Because the format is just text, lyrics from LRC files, notes apps or chat paste straight in, and editing a time means editing a character.

index.htmllines 376–388

function readLine(raw){
  // An optional end time: [1:47-1:58] shows the lyric from 1:47 and clears it at 1:58.
  let end=null;
  raw = raw.replace(RANGE, (x,a,b,c,d)=>{ end=toSec(c,d); return `[${a}:${b}]`; });
  const re=/\[(\d{1,3}):(\d{1,2}(?:[.:]\d{1,3})?)\]/g; const stamps=[]; let m, last=0;
  while((m=re.exec(raw))){ stamps.push(+m[1]*60+parseFloat(m[2].replace(':','.'))); last=re.lastIndex; }
  let text = raw.slice(last).trim();
  if(!stamps.length){
    const p = raw.match(/^\s*(\d{1,3}):(\d{2}(?:\.\d+)?)(?:\s*[-–~]\s*(\d{1,3}):(\d{2}(?:\.\d+)?))?(?:\s+(.*))?$/);
    if(p){ stamps.push(toSec(p[1],p[2])); if(p[3]) end=toSec(p[3],p[4]); text=(p[5]||'').trim(); }
  }
  return {stamps, text, end};
}

A range like [1:47-1:58] is rewritten to a plain start stamp, and the end time is kept aside to clear the screen later.
Every [m:ss] or [m:ss.xx] at the start becomes seconds. A line can carry several stamps, which is how a repeated chorus is written once.
If there are no brackets, a bare 1:07 lyric also works. The parser accepts what people actually type.

index.htmllines 400–411

function parseLyrics(txt){
  const {blocks, orphans} = readBlocks(txt); const lines=[];
  const clears=[];
  for(const b of blocks) for(const s of b.stamps){
    lines.push({time:s, rows:b.rows});
    if(b.end!=null && b.end>b.stamps[0]) clears.push({time:s+(b.end-b.stamps[0]), rows:[], from:s});
  }
  // An end time only clears the screen if no other lyric has started in the meantime.
  for(const c of clears) if(!lines.some(l=>l.time>c.from && l.time<c.time && l.rows.length)) lines.push({time:c.time, rows:[]});
  lines.sort((a,b)=>a.time-b.time);
  return {lines, blocks:blocks.length, untimed:orphans};
}

Each stamp of each block becomes one event: at this time, show these rows.
An end time becomes an empty event, which clears the screen for an interlude.
The clear is dropped if another lyric starts first, so an end time can never blank out the next line by mistake.
Lines before the first timestamp are returned as untimed; the page shows a warning chip for them instead of silently ignoring them.

Time it by tapping along

Your ears do the timing, the page writes the stamps

Most lyrics you find have no times. Retime by tapping plays the song from the start and shows the next block in large type; you press Space when it begins. The lyrics are grouped first (by default, pairs of lines that restart at each section), so one tap times two lines. When you finish, the result is written back into the textarea as ordinary timestamps you can fine-tune.

index.htmllines 1073–1082

const tapTime = () => Math.max(0, now()-.12);
function tap(){ if(!sync) return; const n=nextGroup(); if(n>=sync.groups.length) return; sync.events.push({t:tapTime(), g:n}); syncUI(); if(n+1===sync.groups.length) finishSync(); }
function clearTap(){ if(!sync) return; const last=sync.events[sync.events.length-1]; if(last && last.g<0) return; sync.events.push({t:tapTime(), g:-1}); syncUI(); }
function finishSync(){
  if(!sync) return;
  const out=[];
  for(const e of sync.events){ if(e.g<0){ out.push(fmtStamp(e.t)); continue; } sync.groups[e.g].forEach((l,j)=>out.push(j===0 ? `${fmtStamp(e.t)} ${l}` : l)); }
  for(let i=nextGroup(); i<sync.groups.length; i++) out.push(...sync.groups[i]);
  lyricsEl.value=out.join('\n'); sync=null; syncUI(); refreshLyrics();
}

Every tap is moved 0.12 s earlier, to make up for the delay between hearing a line and pressing the key.
A tap records the time and which group it starts. After the last group, timing finishes on its own.
Pressing C records a group of -1: a clear-screen event. Two clears in a row are ignored.
Each event is written back as text: a lone stamp for a clear, otherwise the stamp on the first line of the group and the other lines under it. That is exactly the format section 02 reads.
Groups you didn't reach are kept, untimed, so stopping halfway loses nothing.

Design choice

Tapping never stores a separate timing file. Writing the result back as text means there is one source of truth, the textarea, and every later edit goes through the same parser.

A scene is a small, checked object

The mood model, Claude and the menus all produce the same shape

A scene is a dozen fields: a setting (hill, beach, city, forest, desert, rainy window, small planet), who sits there, time of day, particles, visualizer, lettering, lyric motion, five colours and a few keywords to highlight. A scene can also carry a mood arc: a list of timed changes to the mood, time of day, particles, visualizer and lyric motion, one per section of the song. Three things can produce one: the Adjust menus, Quick match (the on-device model in the next section), and, when the page runs inside Claude, Claude reading the whole song. All of them pass through one function that only lets valid values through.

index.htmllines 316–338

function normalize(j){
  const t = JSON.parse(JSON.stringify(DEFAULT_THEME));
  if(!j || typeof j !== 'object') return t;
  for(const k of Object.keys(OPTS)) if(typeof j[k]==='string' && OPTS[k][j[k]]) t[k]=j[k];
  if(typeof j.title==='string' && j.title.trim()) t.title=j.title.trim().slice(0,60);
  t.summary = typeof j.summary==='string' ? j.summary.trim().slice(0,220) : '';
  t.campfire = !!j.campfire;
  const base = PALETTES[t.timeOfDay];
  const p = j.palette && typeof j.palette==='object' ? j.palette : {};
  t.palette = {};
  for(const k of Object.keys(base)) t.palette[k] = isHex(p[k]) ? p[k].toLowerCase() : base[k];
  if(contrast(t.palette.text, t.palette.ground) < 4) t.palette.text = lum(t.palette.ground) < .2 ? '#fff8ee' : '#15111c';
  t.keywords = Array.isArray(j.keywords) ? j.keywords.filter(w=>typeof w==='string').map(w=>w.trim().toLowerCase()).filter(w=>w && w.length<=24).slice(0,12) : [];
  t.mood = MOODS[j.mood] ? j.mood : null;
  // Mood arc: [{t, mood, timeOfDay, particles, visualizer, lyricStyle}], sorted by lyric time.
  t.arc = Array.isArray(j.arc) ? j.arc.filter(s=>s && typeof s==='object' && isFinite(s.t)).map(s=>{
    const o = {t:+s.t, mood: MOODS[s.mood] ? s.mood : (t.mood||'tender')};
    for(const k of ARC_KEYS) o[k] = OPTS[k][s[k]] ? s[k] : t[k];
    return o;
  }).sort((a,b)=>a.t-b.t).slice(0,40) : [];
  t.arcOn = !!j.arcOn && t.arc.length > 1;
  return t;
}

Start from the default scene, so anything missing or broken falls back to something that draws.
A field is copied only if its value is one of the known options. An unknown setting name can't reach the renderer.
Each colour must be a 6-digit hex; otherwise the palette for that time of day fills the gap.
A legibility guard: if lyric text and ground colour have a contrast ratio under 4, the text is swapped to near-white or near-black.
The mood arc, one entry per section, goes through the same check: each entry's choices must be known options, and the arc only switches on if it has at least two parts.

If the model can't load, Quick match falls back to the original word rules. They look for words in the lyrics (with timestamps stripped) and map the first theme found to a full scene, for the whole song. Common Chinese words are first translated into the same English cue words, so Chinese lyrics match too.

index.htmllines 350–362

  let setting='hill', particles='fireflies', tod='dusk', campfire=false, visualizer='wave', font='serif', lyricStyle='rise';
  const figures = has(' we ',' us ',' our ','together','hold me','hold you','you and me',"we're",'side by side') ? 'couple' : 'single';
  if(has('rain','storm','tears','cry','grey','gray')){ setting='window'; particles='rain'; tod='night'; font='classic'; lyricStyle='fade'; }
  else if(has('ocean',' sea ','wave','shore','tide','beach','salt','sand')){ setting='beach'; particles='none'; tod='dusk'; visualizer='ring'; }
  else if(has('city','street','neon','traffic','downtown','subway','rooftop','skyline')){ setting='city'; particles='none'; tod='night'; font='condensed'; visualizer='bars'; lyricStyle='karaoke'; }
  else if(has(' fire','burn','flame','smoke','ember','campfire')){ setting='forest'; campfire=true; particles='embers'; tod='night'; font='script'; }
  else if(has('snow','winter','cold','frozen','december',' ice ')){ setting='forest'; particles='snow'; tod='night'; lyricStyle='fade'; }
  else if(has('star','moon','space','galaxy','orbit','planet','heaven')){ setting='space'; particles='stars'; tod='night'; visualizer='ring'; lyricStyle='fade'; }
  else if(has('road','highway','drive','desert','dust','miles','engine')){ setting='desert'; particles='none'; tod='dusk'; font='condensed'; visualizer='bars'; }
  if(has('spring','bloom','flower','petal','cherry','blossom')) particles='petals';
  if(has('morning','sunrise','dawn','wake up')) tod = tod==='night' ? 'dawn' : tod;
  const emotive = ['love','heart','home','forever','alone','stay','light','gone','dream','tonight','remember','together','fire','rain','stars','free','hold','summer','golden','ours','goodbye','never','always'];
  const keywords = emotive.filter(w => new RegExp('\\b'+w+'\\b').test(s)).slice(0,6);

“We”, “us” or “together” in the lyrics puts two people on the bench; otherwise one.
Each rule sets several fields at once, so the choices agree: rain means a window seat at night, a serif display face and a slow fade.
Upbeat cues get energetic choices: condensed capitals, bar visualizer and karaoke fill for a city song.
Some cues layer on top of any setting: blossoms swap in petals, “morning” turns night into dawn.
Up to six emotional words that actually appear in the song become keywords, drawn in the accent colour when they come up.

A small model reads the mood

A 345 KB network distilled from Claude, running in the page

Word rules can only pick one scene for the whole song, and they miss most of it: a chorus about leaving with no rain or ocean in it gets the default hill. So Quick match now runs a small network in the browser. Claude wrote 420 original songs in eight languages and labeled each one with how likely every scene choice was; the network learned to imitate those labels on 360 of them, and the other 60 were held back for testing (model/README.md). The lyrics are first split into sections at blank lines and headings like [Chorus], or at the longer pauses when every line has its own timestamp. The model then predicts the whole song's setting, people and lettering, plus a mood, energy, time of day, particles, visualizer and lyric motion for each section. No API key and no server.

The input side has no vocabulary. Each line becomes hashed character n-grams and whole words, so typos, slang and any language all land somewhere. One model reads English, Chinese, Japanese, Spanish and more. It's strongest in English, where it saw the most songs.

mood.jslines 16–24

function grams(line){
  const t = norm(line); if(!t) return [];
  const out = [], p = cpsOf(' ' + t + ' ');
  for(const n of [2,3,4]) for(let i=0;i+n<=p.length;i++) out.push(p.slice(i,i+n));
  for(const c of cpsOf(t)) if(c >= 0x2E80) out.push([c]);
  for(const w of t.split(' ')) out.push([1, ...cpsOf(w)]);
  return out;
}
function feats(lines, B){ const m = B-1, out = []; for(const l of lines) for(const g of grams(l)) out.push(fnv(g) & m); return out; }

The line is normalised (NFKC, lower case, letters and digits only) and padded with spaces, so n-grams at the edges mark word starts and ends.
Character 2-, 3- and 4-grams: “heart” shares most of its grams with “hearts” and “heartbeat”.
Chinese, Japanese and Korean characters also count one by one, since a single character often carries the meaning.
Whole words too, tagged with a leading 1 so they never collide with a 2-gram of the same letters.
Each feature is hashed (32-bit FNV-1a) into one of 8,192 buckets, which index a 32-dimensional int8 embedding table.

A section is the mean of its features' embeddings. Its head sees more than the section itself: the whole song, its neighbours, where it sits and whether its lines repeat elsewhere, which is a strong hint that it's a chorus.

mood.jslines 86–96

    const secs = sections.map((s, i) => {
      const ctx = new Float32Array(D); const nb = [i-1, i+1].filter(j => j>=0 && j<n);
      for(const j of nb) for(let d=0;d<D;d++) ctx[d] += Sv[j][d]/nb.length;
      const others = new Set(normed.flatMap((o,j)=> j===i ? [] : o));
      const rep = normed[i].filter(l => l && others.has(l)).length / Math.max(1, normed[i].length);
      const ex = Float32Array.from([n>1 ? i/(n-1) : 0, i===0?1:0, i===n-1?1:0, rep]);
      const Hs = dense(cat(Sv[i], G, ctx, ex), 'Ws', 'bs', true);
      const out = {};
      for(const [k, opts] of Object.entries(head.secHeads)) out[k] = named(opts, softmax(dense(Hs, 'W_'+k, 'b_'+k)));
      return out;
    });

Context: the average of the sections just before and after.
The share of this section's lines that also appear elsewhere in the song. Choruses repeat; verses mostly don't.
Position in the song, and flags for first and last section.
Section, whole song, neighbours and position go through one 64-unit ReLU layer.
One softmax per choice, so the page gets a probability for every option, not just a winner.

Section-by-section predictions are noisy, and a scene that changes every 15 seconds looks broken. The arc is smoothed and given hysteresis before it's used.

index.htmllines 474–491

function moodArc(pred, secs){
  const n=secs.length, heads=['mood',...ARC_KEYS];
  const margin={mood:.12, timeOfDay:.25, particles:.18, visualizer:.2, lyricStyle:.2};
  const pick={};
  for(const h of heads){
    const P=pred.sections.map(s=>s[h]);
    const sm=P.map((p,i)=>{ const o={}; for(const k in p){ let v=.7*p[k], w=.7; for(const j of [i-1,i+1]) if(j>=0&&j<n){ v+=.15*P[j][k]; w+=.15; } o[k]=v/w; } return o; });
    const c=[]; sm.forEach((p,i)=>{ const a=argmax(p); c.push(i && p[a]-p[c[i-1]] < margin[h] ? c[i-1] : a); });
    pick[h]=c;
  }
  const arc=[];
  for(let i=0;i<n;i++){
    const s={t:secs[i].t}; for(const h of heads) s[h]=pick[h][i];
    const last=arc[arc.length-1]; if(last && heads.every(h=>last[h]===s[h])) continue;
    arc.push(s);
  }
  return arc;
}

How much better the new choice has to be before the scene changes. Time of day, the most noticeable change, needs the widest margin.
Each section's probabilities are blended with its neighbours' (70 / 15 / 15).
Hysteresis: keep the previous section's choice unless the new favourite beats it by the margin.
Sections that come out identical merge, so the arc only lists real changes.

At playback the change is a crossfade, not a cut. The palette is blended colour by colour, and particles and visualizers fade between the two looks.

index.htmllines 906–922

const ARC_FADE = 2.4, ARC_LEAD = 1.2;
let mixState = null; // {cur, prev, k, j} for the frame being drawn
function arcState(lt){
  const a = theme.arc;
  if(!theme.arcOn || !a || a.length<2) return null;
  let j=0; for(let i=1;i<a.length;i++) if(lt >= a[i].t - ARC_LEAD) j=i;
  const cur=a[j], prev=j ? a[j-1] : null;
  return {cur, prev, k: prev ? clamp((lt-(cur.t-ARC_LEAD))/ARC_FADE) : 1, j};
}
function sceneView(lt){
  const st = arcState(lt);
  if(!st) return {view:theme, st:null};
  const {cur, prev, k} = st;
  let P = moodPalette(cur.timeOfDay, cur.mood);
  if(prev && k<1){ const Q = moodPalette(prev.timeOfDay, prev.mood), e = ease(k); P = Object.fromEntries(Object.keys(P).map(x=>[x, mix(Q[x], P[x], e)])); }
  return {view:{...theme, palette:P, mood:cur.mood, timeOfDay:cur.timeOfDay, particles:cur.particles, visualizer:cur.visualizer, lyricStyle:cur.lyricStyle}, st};
}

Each section's look starts 1.2 s before its first lyric and takes 2.4 s to arrive, so the change lands with the singing.
Find the section in effect, counting the lead-in.
k runs from 0 to 1 across the fade.
Each section's colours come from its time of day and mood. During a fade the two palettes are mixed with an eased k.
The frame is drawn from this blended view. The saved scene itself never changes.

The strip under the player shows the arc, one segment per stretch of the song coloured by its mood; click a part to jump to it. If you change the time of day, particles, visualizer, motion or colours by hand, that look holds for the whole song until you tick the box again.

How well it does

On the 60 held-out songs, 82% of the model's choices were ones Claude rated as reasonable, compared with 53% for the old word rules. Broken down: on those songs written in different voices, Claude would have accepted the model's choice (given it at least 20% probability) for 82–84% of particles, visualizers, lyric motion and lettering, against 39–60% for the word rules. The setting is the weak spot (70%). JavaScript and Python give the same probabilities to within 10⁻⁶, checked by a parity script.

Paint every frame from scratch

One canvas, one loop, lettering that always fits

Everything on the stage is drawn with the 2D canvas API, redrawn on every display frame by one function, which both playback and export call. There are no image files: hills, buildings, pines and the people on the bench are paths and gradients built from the scene's five colours. Random details like star positions come from a seeded generator, so the same scene looks the same each time it loads.

index.htmllines 964–983

function frame(ts){
  const dt=Math.min(.05,(ts-last)/1000); last=ts;
  if(!hasAudio && playing){ demoT+=dt; if(demoT>=duration()){ demoT=duration(); setPlaying(false); } }
  const t=now(), anim=ts/1000;
  if(!(recording && recording.fast)) renderFrame(t, anim, dt); // fast export draws its own frames
  updateTransport(t);
  requestAnimationFrame(frame);
}
// Draw one frame: t is song time, anim drives ambient motion, dt is the frame step.
function renderFrame(t, anim, dt, off){
  analyse(t, dt, off);
  const base = theme, sv = sceneView(t - offset);
  theme = sv.view; mixState = sv.st;
  try{
    drawScene(anim, au);
    drawParticles(anim, au);
    drawViz(au);
    drawLyrics(t, au);
  } finally { theme = base; mixState = null; }
}

Frame time is capped at 50 ms, so a background tab coming back doesn't make rain jump across the screen.
Two clocks: t is the song position (it drives lyrics and the mood arc), anim is wall time (it keeps fireflies drifting even while paused).
During fast export the export loop draws its own frames, so the live loop stays out of the way.
The audio is analysed first, so the moon's glow, the bars and the embers can all react to the same bass, mid and treble levels.
The scene for this moment, mid-crossfade if need be, temporarily stands in for the saved one.
Painter's order, back to front: sky and scene, particles, visualizer, then the lyrics. finally puts the saved scene back even if a draw fails.

The lyrics are where motion and type meet. A block of two long lines in a big display face can overflow the frame, so the size is found by trying, and the chosen motion style turns elapsed time into how much of the text is visible.

index.htmllines 849–863

  // Fit the whole block: shrink until it fits in the lower part of the frame.
  let size = Math.round(F.size*(isCJK?F.cjk:1)), rows=[], maxW=W*.8;
  for(let attempt=0; attempt<5; attempt++){
    ctx.font=F.f(size); rows=[]; let c=0;
    for(const tx of texts){ const r=wrapText(tx, maxW, c); rows.push(...r.rows); c=r.end+1; }
    if(rows.length*size*1.25 <= H*.36 || size<26) break;
    size=Math.round(size*.86);
  }
  const lh=size*(isCJK?1.38:1.16), base=H*.91 - (rows.length-1)*lh;
  const style = styleAt(L.time); // a line keeps its section's motion even while the scene crossfades
  const rise = style==='rise' ? (1-ease(aIn))*24 : 0;
  const total = full.length;
  const typed = style==='typewriter' ? Math.floor(total*clamp(el/Math.min(2.4,dur*.55))) : Infinity;
  const sung = style==='karaoke' ? clamp(el/(dur*.88))*total : Infinity;
  const mask = keywordMask(full);

The starting size depends on the lettering, scaled down for Chinese, Japanese or Korean text, whose glyphs are denser.
Wrap at 80% of the width, check the block fits in the lower 36% of the frame, and if not shrink by 14% and try again, at most five times.
“Rise in”: the text starts 24 px low and eases up as it fades in. The motion comes from the line's own section, so it doesn't change halfway through a line.
“Typewriter” reveals characters over at most 2.4 s; “Karaoke” fills them in across the time until the next lyric.
The keyword mask marks which characters belong to a highlighted word, for both English words and Chinese phrases.

Record what you see, keep the song at home

Frame-by-frame encoding, with live recording as the fallback

Where the browser has WebCodecs (recent Chrome, Edge and Safari), export doesn't play the song at all. It decodes the song, then draws every frame with the same renderFrame and hands it to the browser's encoder through the small Mediabunny library, usually much faster than real time.

index.htmllines 1225–1233

  const off=offlineAnalyser(song, FPS); buildWorld();
  const t0=performance.now(); let drawMs=0;
  for(let i=0;i<n;i++){
    if(recording.cancel){ await output.cancel(); return null; }
    const t=i/FPS, d0=performance.now(); renderFrame(t, t, 1/FPS, off); drawMs+=performance.now()-d0;
    await video.add(t, 1/FPS);
    if(i%6===0){ recBtn.querySelector('span').textContent=`Exporting… ${Math.min(99,Math.floor(i/n*100))}%`; await nextTick(); }
  }
  video.close(); await output.finalize();

There is no live audio to measure, so an offline analyser reproduces the browser's own (1024-point FFT, Blackman window, same smoothing) on the decoded song. The visualizer looks the same as during playback.
Frame i is drawn at song time i/30, through the same function playback uses.
The canvas is encoded as a frame with an exact timestamp, so picture and sound can't drift.

Elsewhere, or if the fast path fails, the page falls back to recording: it plays the song once from the start while MediaRecorder records the canvas and the audio together. The trick there is wiring the audio so the same signal reaches the speakers, the visualizer and the recorder.

index.htmllines 999–1007

function ensureAudio(){
  if(actx) return;
  actx = new (window.AudioContext||window.webkitAudioContext)();
  const src = actx.createMediaElementSource(audio);
  analyser = actx.createAnalyser(); analyser.fftSize=1024; analyser.smoothingTimeConstant=.78;
  src.connect(analyser); analyser.connect(actx.destination);
  recDest = actx.createMediaStreamDestination(); analyser.connect(recDest);
  freq = new Uint8Array(analyser.frequencyBinCount); wave = new Uint8Array(analyser.fftSize);
}

The <audio> element is routed into Web Audio, so its sound can be measured and copied.
An analyser with a 1024-point FFT gives the frequency and waveform data the visualizers draw.
One branch goes to the speakers, as normal playback.
A second branch goes to a MediaStreamDestination, which turns the audio into a track a recorder can use.

index.htmllines 1273–1280

function startRecording(){
  ensureAudio();
  const types=['video/mp4;codecs=avc1.42E01E,mp4a.40.2','video/mp4','video/webm;codecs=vp9,opus','video/webm;codecs=vp8,opus','video/webm'];
  const mime=types.find(m=>MediaRecorder.isTypeSupported(m))||'';
  const stream=cv.captureStream(30);
  const at=recDest.stream.getAudioTracks()[0]; if(at) stream.addTrack(at);
  let rec; try{ rec=new MediaRecorder(stream, mime?{mimeType:mime, videoBitsPerSecond:8e6}:{}); }catch(e){ $('#recNote').textContent='Export couldn’t start in this browser.'; return; }
  const chunks=[]; rec.ondataavailable=e=>{ if(e.data.size) chunks.push(e.data); };

Ask for MP4 first and fall back to WebM, so the file plays in most places when the browser can write it.
The canvas becomes a 30 fps video stream: exactly what the preview shows, lyrics included.
The audio track from the snippet above is added to the same stream, so picture and sound stay in sync.
Recorded at 8 Mbit/s. If the browser can't record at all, the page says so instead of failing silently.

Privacy

The song is opened with URL.createObjectURL, a link to the file that exists only in this tab. Lyrics, song title and scene are remembered in localStorage on your device, wrapped in try/catch so a private window still works. Nothing is sent to a server, and the exported video is built in your browser too.

→

Read further

Where everything lives

index.html the app: layout and CSS, parser, tap-sync, scenes and mood arc, canvas renderer, audio analysis and export
mood.js the on-device model: features, int8 embeddings, forward pass (weights in mood-model.bin)
model/ training data, NumPy training with hand-written gradients, parity checks and results

Back to Liner Lyric Studio All projects