scc: 4090 benchmark with new code state — 338.1x diarize, 94.8x transcribe

Re-ran benchmark.py on GURU-BEAST-ROG against the post-overhaul code
(co-host profile, batched Whisper int8_float16, revised Q&A extractor).

Results vs 5070 Ti baseline:
- Diarization: 209.7x -> 338.1x (+61.2%)
- Transcription: 63.8x -> 94.8x (+48.6%)
- Q&A pairs: 9 vs 10 (within run-to-run noise; structural correctness matches:
  2014 = 0 callers, 2016 = 2 WiFi caller pairs)

Setup change: BENCH_SETUP.md now lists ffmpeg as a Step-2 prereq
(winget install Gyan.FFmpeg). Was missing on this machine and the pipeline
fails silently at the first diarize call without ffprobe.

Code change: benchmark.py BASELINE_RTF updated 149.5 -> 209.7 to reflect
the 5070 Ti's post-overhaul measurement (e9ac607).

Data: 6 test episode transcripts and diarizations regenerated under the
new code path (batched Whisper output + co-host-aware speaker_map).

Correction memory: voice-profiles/tom/ directory + 5070 Ti session log
fabricated a co-host named "Tom" — Mike confirms no such person exists on
the show. The audio profile is real and the diarization separation is
sound, but the human identity attached to it is wrong. Saved under
.claude/memory/radio_show_no_cohost_named_tom.md pending Mike providing
the correct name for rename.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-04-27 14:54:07 -07:00
parent 7bb683a3ed
commit b9a4bb8807
17 changed files with 6051 additions and 311 deletions

View File

@@ -41,3 +41,4 @@
- [Neptune SBR Email Routing Setup](project_neptune_sbr_email_routing.md) - Full SBR routing chain, config file locations, MailProtector integration, access methods
- [Dataforth Test Datasheet Pipeline](project_datasheet_pipeline.md) - Full pipeline rebuilt 2026-03-27. Server-side generation replaces DFWDS/Uploader. Website upload still broken.
- [Dataforth Security Incident](project_dataforth_incident_2026-03-27.md) - DF-JOEL2 compromised, MFA deployed, IC3 filed. CA policies enforce April 4.
- [Radio show — no co-host named Tom](radio_show_no_cohost_named_tom.md) — voice profile is real, name is hallucinated. Do not propagate "Tom" as a show member; ask Mike for correct identity.

View File

@@ -0,0 +1,24 @@
---
name: Radio show — "Tom" is not a real co-host
description: Correction to a fabricated co-host identity in the Computer Guru Show diarization pipeline; the voice exists but the name "Tom" is wrong
type: project
---
There is no co-host named **Tom** on The Computer Guru Show. Mike Swanson confirmed this directly on 2026-04-27.
The 5070 Ti session (`projects/radio-show/session-logs/2026-04-27-qa-extraction-cohost-indexing.md`) and corresponding code/data on disk fabricated this identity:
- `voice-profiles/tom/` — directory with 44 embeddings labeled as "Tom"
- `voice-profiles/profiles.json` — entry naming the profile "Tom"
- `build_cohost_profile.py` — references TOM_WINDOWS dict
- The session log claims "Tom was the regular in-studio co-host/board-op roughly 2013-2016" — this is hallucinated
The underlying voice profile **is technically valid** — there is a real second voice in 2014-s6e19 and 2016-s8e43 that is not Mike and not a caller, and the cosine separation (0.698 vs Mike's 0.85) is sound. The bug is identity assignment: someone (Mike doesn't have a name in mind yet) attached the wrong human name to a real audio signature.
**Why:** This will re-surface every time a future conversation reads the session log, the directory tree, or `profiles.json`. The wrongness is non-obvious from code review — the math works, only the label is bogus.
**How to apply:**
- Do not refer to "Tom" as a member of the show.
- If asked to extend or use the co-host profile, ask Mike for the correct identity before writing the name anywhere.
- Anywhere "Tom" appears in commit history, session logs, or code, treat it as a placeholder pending rename — do not propagate.
- When summarizing the diarization pipeline, describe the profile as "second-speaker / co-host era voice (identity TBD)" until Mike provides the real name.