How a walkable album holds together

Kanye made an album he had no business making. He couldn’t sing, so he sang through an autotune box and built one of the most-listened-to records of the decade. I can’t make music. I can’t sing. But the generative audio tools got good enough this year that I could write lyrics, hand them to an AI, and get back songs I actually liked listening to. So I made an album.

“H100s” for the GPUs doing the singing for me. “Burn Rate” for the founder story the lyrics keep circling around — late nights, rewrites, the thing you can’t quite pay for yet. Same structure as “808s & Heartbreak”: the tool I couldn’t use, paired with the feeling I couldn’t shake. A world of songs from someone who can’t make music.

The early prototype was a playlist. I dropped it the first afternoon. The thing that made the album feel like a world was letting every zone play simultaneously, mixed by distance. Walk away and it fades. Walk into the center and everything else mutes so you can actually listen.

Each zone gets an HTMLAudioElement plumbed through its own GainNode on a single AudioContext. Every frame the update loop ramps each source’s gain toward a target based on distance and solo state.

// Inverse-distance falloff (WebAudio "inverse" model) — real-world-like.
if (distance <= this.minDistance) {
  volume = 1;
} else if (distance < this.maxDistance) {
  volume =
    this.minDistance /
    (this.minDistance + this.rolloffFactor * (distance - this.minDistance));
  // Soft fade to zero near the outer edge so it doesn't pop
  const fadeStart = this.maxDistance * 0.85;
  if (distance > fadeStart) {
    const fade = 1 - (distance - fadeStart) / (this.maxDistance - fadeStart);
    volume *= Math.max(0, fade);
  }
}

// When inside a zone, solo that song — pure listening
if (this.soloId) {
  volume = source.songId === this.soloId ? 1 : 0;
}

source.gainNode.gain.linearRampToValueAtTime(
  volume,
  this.ctx.currentTime + 0.15
);

Every zone sits within the spawn’s 600-pixel load radius. Naive implementation: the scene mounts, twenty-plus new Audio() calls fire simultaneously, the browser parallelizes them all to a single Vercel Blob origin, and everyone gets stuck waiting.

Fix: collect everything in range, sort nearest-first, throttle in-flight loads to four. Slower songs wait their turn and get picked up on subsequent ticks as earlier ones reach HAVE_ENOUGH_DATA.

// Start loads nearest-first, throttled so the browser doesn't fan out 19
// cold fetches to one origin in parallel. Songs whose turn hasn't come
// yet will get picked up on a later tick once in-flight loads finish.
pendingLoads.sort((a, b) => a.distance - b.distance);
let budget = this.maxConcurrentLoads - this.inFlightLoadCount();
for (const { meta } of pendingLoads) {
  if (budget <= 0) break;
  this.loadSong(meta);
  budget--;
}

Nearest song is playable in a second or two on a fresh load; the far edge of the spawn follows seconds later. No more “new joiner hears silence.”

Every song ends up in three forms. A local script transcodes the mp4 with ffmpeg and pushes each tier to Vercel Blob.

• Preview mp3 — 48 kbps mono, ~670 KB. What spatial audio streams in the world.
• Full-res mp3 — 192 kbps stereo, ~3 MB. The modal player and the playlist.
• MP4 original— kept as a fallback for anything new that hasn’t finished transcoding yet. Never hit on the happy path.

This was the single biggest win. A new joiner walking through every zone pulls ~15 MB of audio now; the pre-optimization version was pulling ~250 MB of mp4.

I originally ran whisper-1for both lyrics and VTT. On one mostly-instrumental track it produced a repeating cue “We are turning right to Session Road.” It was not about Session Road.

The new pipeline uses gpt-4o-transcribe for the text (much higher resistance to musical hallucinations), and keeps whisper-1only for the VTT timings — because gpt-4o-transcribe still doesn’t emit timestamps on the non-streaming endpoint. A word-overlap check catches whisper when it spirals, and falls back to synthesizing a VTT by spreading gpt-4o’s accurate lines across the song’s duration.

const vttCueText = whisperVttText
  .split("\n")
  .filter(l => l && !l.startsWith("WEBVTT") && !/-->/.test(l) && !/^\d+$/.test(l))
  .join(" ");
const vttWords = wordSet(vttCueText);
const lyricWords = wordSet(lyrics);
let overlap = 0;
for (const w of vttWords) if (lyricWords.has(w)) overlap++;
const overlapRatio = vttWords.size > 0 ? overlap / vttWords.size : 0;

if (overlapRatio < 0.3 && vttWords.size > 0) {
  // whisper hallucinated. Build the VTT ourselves from accurate lyrics.
  writeSynthesizedVtt(vttPath, lyrics, song.duration);
} else {
  fs.writeFileSync(vttPath, whisperVttText, "utf-8");
}

Every zone ring is a Phaser Graphics object that redraws from a state machine. Idle: dim full circle. Loading: a partial arc that sweeps around on a tween counter. Ready: solid stroke at full color. The ring transitions live with the load state emitted by SpatialAudio.

if (v.state === "ready") {
  g.lineStyle(2, 0x4a90d9, 0.55);
  g.strokeCircle(0, 0, 48);
} else if (v.state === "loading") {
  // A two-thirds arc that sweeps around — reads as "working on it".
  g.lineStyle(2, 0x4a90d9, 0.4);
  const start = v.ringAngle;
  const end = start + Math.PI * (4 / 3);
  g.beginPath();
  g.arc(0, 0, 48, start, end, false);
  g.strokePath();
} else {
  // idle — faint hint the zone is there but not yet ready.
  g.lineStyle(1.5, 0x4a90d9, 0.2);
  g.strokeCircle(0, 0, 48);
}

The world itself is a Phaser scene. Everything wrapping it — splash, chat, marker input, song modal, minimap, menu — is React. The canvas component is dynamically imported with ssr: false so the Phaser bundle never touches the server.

The two sides talk through a handful of window.__h100s*functions. React sets active song, pushes stats, shows bubbles; Phaser records dances, pulls chat pool, notifies on approach. Ugly in the abstract. Pragmatic at this size. Lets Phaser’s 60fps render loop and React’s reconciler run independently without anyone stepping on a state update.

// React -> Phaser
win.__h100sSetSongTimes(songTimesRef.current);
win.__h100sSetSongDances(songDancesRef.current);
win.__h100sShowChatBubble("self", text);

// Phaser -> React
win.__h100sOnDance = (pos) => {
  pendingDances += 1;
  if (activeSongId) pendingPerSongDances[activeSongId]++;
  realtimeRef.current?.publishEvent({ type: "dance", ...pos });
};

Presence is Ably. Every other connected player shows up as a tinted sprite at their last-known position, interpolated toward the next update. Chat bubbles come through as events; the same speech-bubble renderer serves local typing, remote players, and NPC bots. The bot pool is generated lines plus real user chats from the history, so even when you’re alone the world is overheard.

Every fifteen seconds the client flushes listen seconds and dance counts to /api/stats. Each increment is a single pipelined round-trip of INCRBY + HINCRBY calls, so two concurrent writers can never clobber each other. A SETNX-gated migration reads an older Vercel Blob JSON blob exactly once and seeds Redis — everyone after the first deploy is a no-op.

const pipe = redis.pipeline();
const perSongTotal = Object.values(delta.perSong).reduce((a, b) => a + b, 0);
if (perSongTotal > 0) pipe.incrby(K.totalListen, perSongTotal);
if (delta.dances > 0) pipe.incrby(K.totalDances, delta.dances);
for (const [songId, secs] of Object.entries(delta.perSong)) {
  if (secs > 0) pipe.hincrby(K.perSong, songId, secs);
}
for (const [songId, count] of Object.entries(delta.perSongDances ?? {})) {
  if (count > 0) pipe.hincrby(K.perSongDances, songId, count);
}
await pipe.exec();

Earlier versions rendered every zone label as 0s | 0 on first frame and only updated once a client-side fetch to /api/statslanded. For songs the current user hadn’t walked past yet, this was visible. page.tsx is now force-dynamicand reads Redis on the server, baking the listen and dance counts into each song’s data. The scene renders the right label on the first frame; the client-side fetch runs for freshness but almost never changes what you see.

let initialStats = { perSong: {}, perSongDances: {} };
if (isConfigured()) {
  try {
    const s = await readStats();
    initialStats = { perSong: s.perSong, perSongDances: s.perSongDances };
  } catch { /* leave empty on transient errors */ }
}

const publishedSongs = await Promise.all(
  songs.filter(s => s.status === "published" && s.zonePosition).map(async s => ({
    // ... id, title, position, urls, vtt, story ...
    listenSeconds: initialStats.perSong[s.id] ?? 0,
    danceCount:    initialStats.perSongDances[s.id] ?? 0,
  }))
);