azcomputerguru/claudetools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

radio: rename Tom -> Tara, expand speaker roster

Browse Source 
Mike confirmed there is no co-host named "Tom" — the voice in 2014-s6e19
and 2016-s8e43 is Tara. The 5070 Ti session fabricated the Tom identity.
The voice profile itself (44 embeddings, 0.698 cosine vs Mike) is correct;
only the human label was wrong.

Rename swept:
- voice-profiles/tom/ -> voice-profiles/tara/ (git mv preserves all .npy)
- voice-profiles/profiles.json: "Tom" key -> "Tara"
- build_cohost_profile.py: TOM_WINDOWS -> TARA_WINDOWS, COHOST_NAME, comments
- 2026-04-27-qa-extraction-cohost-indexing.md: correction header + body sweep
- 2026-04-27-4090-benchmark-and-test-set.md: closure note
- .claude/memory/radio_show_no_cohost_named_tom.md: resolution + speaker roster

Diarization re-run after rename so speaker_map emits "Cohost: Tara".
Q&A counts unchanged (rename is label-only): 9 pairs across 6 test episodes.

Tara distribution from the post-rename diarization (per-episode % of audio):
  2011-03-12-hr1   140s   5.6%   likely false positive (call-in only)
  2012-03-10-hr1    30s   1.1%   likely false positive (call-in only)
  2012-06-09-hr1   340s  12.8%   suspicious — pending Mike confirm
  2014-s6e19       680s  23.3%   confirmed
  2016-s8e43      1890s  35.5%   confirmed
  2017-s9e30       610s  11.4%   plausible — pending Mike confirm

Broader speaker-roster context Mike provided this session (saved to
memory): the show has had multiple co-hosts (Tara, Randall, Rob) plus
producers/board ops (Andrew, Shannon, Ken, others) who would sometimes
go on-air. Only Tara has a profile so far. Every other speaker is
currently labeled CALLER, which means small CO-HOST attributions in
unexpected episodes (e.g. 2011/2012) may actually be a producer rather
than a false positive — Mike to spot-check.

Action item before full-archive run: build profiles for Randall, Rob,
and the named producers to avoid systematic Q&A false positives in
early-years and 2018/2019 episodes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Mike Swanson
2026-04-27 15:11:03 -07:00
parent b9a4bb8807
commit fb683d6a05
55 changed files with 122 additions and 53 deletions
Download Patch File Download Diff File Expand all files Collapse all files

40
projects/radio-show/session-logs/2026-04-27-qa-extraction-cohost-indexing.md
View File

@@ -2,6 +2,14 @@
**Date:** 2026-04-27
**Project:** Radio Show Archive Mining — Computer Guru Show
> **Correction (2026-04-27, GURU-BEAST-ROG session):** This log was originally
> written referring to the co-host as "Tom." Mike confirmed there is no co-host
> by that name; the voice in 2014-s6e19 and 2016-s8e43 is **Tara**. The voice
> profile is correct (clean 0.698 cosine separation from Mike), only the human
> identity attached to it was wrong. All references below have been updated
> Tom → Tara. There have been multiple co-hosts on the show over the years;
> Tara is one of them.
---
## User
@@ -13,9 +21,9 @@
## Session Summary
The session began with resuming work following a benchmark run that demonstrated a significant performance improvement in Whisper transcription, achieving 63.8x real-time speed with batched inference and int8_float16 settings. Next, the focus shifted to evaluating the quality of Q&A extraction across six test episodes, revealing a critical issue with false positives due to co-host Tom being mislabeled as CALLER based on a voice similarity threshold.
The session began with resuming work following a benchmark run that demonstrated a significant performance improvement in Whisper transcription, achieving 63.8x real-time speed with batched inference and int8_float16 settings. Next, the focus shifted to evaluating the quality of Q&A extraction across six test episodes, revealing a critical issue with false positives due to co-host Tara being mislabeled as CALLER based on a voice similarity threshold.
A co-host voice profile for Tom was constructed using 44 embeddings from two specific episodes (2014-s6e19 and 2016-s8e43), producing a cosine similarity of 0.698 against Mike — well below Mike's 0.85 threshold, giving clean separation. Code was updated in `voice_profiler.py` and `diarizer.py` to correctly emit "Cohost: Tom" labels and map them to a new "CO-HOST" speaker tag. Re-diarizing the two co-host-era episodes dramatically cleaned up Q&A results: 2016 went from 12 false positives to 2 real WiFi caller pairs.
A co-host voice profile for Tara was constructed using 44 embeddings from two specific episodes (2014-s6e19 and 2016-s8e43), producing a cosine similarity of 0.698 against Mike — well below Mike's 0.85 threshold, giving clean separation. Code was updated in `voice_profiler.py` and `diarizer.py` to correctly emit "Cohost: Tara" labels and map them to a new "CO-HOST" speaker tag. Re-diarizing the two co-host-era episodes dramatically cleaned up Q&A results: 2016 went from 12 false positives to 2 real WiFi caller pairs.
Several bugs in `qa_extractor.py` were fixed: overlap resolution for sliding-window diarization boundaries, CALLER-preference threshold for long batch transcript segments, and a turn-based caller-intro lookback to replace an ineffective 120s time window. Phone-greeting detection and new promo signatures were added. The final Q&A count landed at 10 pairs across 6 episodes, with 2014 correctly yielding 0 (gaming co-host episode with no actual callers).
@@ -25,21 +33,21 @@ Several bugs in `qa_extractor.py` were fixed: overlap resolution for sliding-win
## Key Decisions
- **Co-host threshold uses same 0.85 bar as host**: Tom scores 0.698 vs Mike. Any voice >= 0.85 against Tom's composite gets labeled CO-HOST. Keeps the same single threshold for all profiles rather than per-profile thresholds.
- **Co-host threshold uses same 0.85 bar as host**: Tara scores 0.698 vs Mike. Any voice >= 0.85 against Tara's composite gets labeled CO-HOST. Keeps the same single threshold for all profiles rather than per-profile thresholds.
- **Turn-based lookback for caller-intro (2 HOST turns, not 120s)**: Long HOST monologue blocks (8-10 min) in big show segments meant time-based lookback missed the caller introduction. Previous 2 HOST turns always catches it regardless of block length.
- **CALLER-preference at 4s minimum overlap**: Batch transcription produces ~26s segments; diarization CALLER windows are ~10s. Pure majority-vote always gave HOST. 4s minimum CALLER coverage labels the segment CALLER without being overly aggressive for co-host episodes.
- **Midpoint boundary resolution at load time**: Rather than re-diarizing everything, the sliding-window overlap is resolved in `load_diarized_transcript()` so it applies retroactively to all saved diarization files without touching the JSON.
- **751-1041 added as promo signal**: Earlier Tucson show number (vs 790-2040 in later seasons). Weighted 1 (needs a second semi-generic signal to filter).
- **Tom's windows sourced from first 60 min of co-host episodes**: Real callers don't call in during the first hour of a 2-hour show (only exceptions: very end of show). First-hour CALLER windows are safely all Tom.
- **Tara's windows sourced from first 60 min of co-host episodes**: Real callers don't call in during the first hour of a 2-hour show (only exceptions: very end of show). First-hour CALLER windows are safely all Tara.
---
## Problems Encountered
- **2016-s8e43 had 12 Q&A pairs, 11 false positives**: Root cause was Tom (co-host) labeled CALLER throughout. Fixed by building Tom's voice profile and re-diarizing.
- **2016-s8e43 had 12 Q&A pairs, 11 false positives**: Root cause was Tara (co-host) labeled CALLER throughout. Fixed by building Tara's voice profile and re-diarizing.
- **2014-s6e19 had 2 Q&A pairs from gaming discussion**: Same co-host issue. After re-diarization: 0 pairs (correct — no actual callers in that gaming special).
- **2012-03-10 yielded 0 segments labeled CALLER**: Midpoint assignment hit HOST turns (HOST 0-20s and CALLER 15-30s — midpoint 15.1s falls in HOST). Fixed by overlap-preference assignment with 4s CALLER minimum.
- **Real WiFi caller (2016, ~4794s) was missing after first fix attempt**: Aggressive time-based lookback (120s) combined with short CALLER turns from sliding-window diarization caused the caller question to land in a HOST segment. Fixed by turn-based lookback + co-host profile (eliminated Tom noise, letting real caller windows survive).
- **Real WiFi caller (2016, ~4794s) was missing after first fix attempt**: Aggressive time-based lookback (120s) combined with short CALLER turns from sliding-window diarization caused the caller question to land in a HOST segment. Fixed by turn-based lookback + co-host profile (eliminated Tara noise, letting real caller windows survive).
- **2012-Jun pair at 1325s was a promo**: "The Computer Guru. We'll get your problem solved. Call 751-1041 today" passed promo filter. Fixed by adding 751-1041 and "we'll get your problem solved" as promo signatures.
---
@@ -51,8 +59,8 @@ Several bugs in `qa_extractor.py` were fixed: overlap resolution for sliding-win
projects/radio-show/audio-processor/build_cohost_profile.py
projects/radio-show/audio-processor/index_test_episodes.py
projects/radio-show/audio-processor/archive.db
projects/radio-show/audio-processor/voice-profiles/tom/
projects/radio-show/audio-processor/voice-profiles/profiles.json (updated: Tom added)
projects/radio-show/audio-processor/voice-profiles/tara/
projects/radio-show/audio-processor/voice-profiles/profiles.json (updated: Tara added)
projects/radio-show/session-logs/2026-04-27-qa-extraction-cohost-indexing.md (this file)
```
@@ -63,8 +71,8 @@ src/diarizer.py — map "Cohost:" prefix to "CO-HOST" speaker
src/qa_extractor.py — overlap resolution, CALLER-preference, turn-based
caller-intro lookback, _preceded_by_caller_intro(),
_PHONE_GREETING, 751-1041 + promo sig additions
test-data/transcripts/2014-s6e19/diarization.json (re-diarized with Tom profile)
test-data/transcripts/2016-s8e43/diarization.json (re-diarized with Tom profile)
test-data/transcripts/2014-s6e19/diarization.json (re-diarized with Tara profile)
test-data/transcripts/2016-s8e43/diarization.json (re-diarized with Tara profile)
```
---
@@ -118,11 +126,11 @@ Q&A pairs: 10
| Name | Role | Embeddings | Source Episodes |
|------|------|-----------|-----------------|
| Mike Swanson | host | 180 | 9 episodes (2010-2018) |
| Tom | cohost | 44 | 2014-s6e19, 2016-s8e43 |
| Tara | cohost | 44 | 2014-s6e19, 2016-s8e43 |
Tom vs Mike cosine similarity: **0.698** (well-separated at 0.85 threshold)
Tara vs Mike cosine similarity: **0.698** (well-separated at 0.85 threshold)
**Tom's source windows used:**
**Tara's source windows used:**
- 2014-s6e19: 195-260s, 320-425s, 600-650s, 675-710s
- 2016-s8e43: 100-115s, 135-160s, 270-295s, 575-605s, 1185-1235s, 1790-1870s, 2020-2055s
@@ -130,7 +138,7 @@ Tom vs Mike cosine similarity: **0.698** (well-separated at 0.85 threshold)
## Co-Host Era Notes
Tom was the regular in-studio co-host/board-op roughly 2013-2016. His voice is in episodes from at least 2014 through 2016 (confirmed from test set). The 2011 and 2012 episodes are pure call-in format with no co-host.
Tara was an in-studio co-host whose voice appears in 2014-s6e19 and 2016-s8e43 (confirmed by Mike). The 2011 and 2012 episodes are pure call-in format with no co-host. Mike notes the show has had multiple co-hosts over the years; Tara's exact tenure isn't fixed from the original 2013-2016 assumption — that should be verified before generalizing the profile across the full archive.
If there are occasional guest co-hosts or fill-in hosts in other years, they would still be labeled CALLER until profiled. These would be rare and would likely not form question patterns that survive the caller-intro gate.
@@ -190,9 +198,9 @@ The pipeline is idempotent — `add_segments()` skips episodes already indexed.
### 4. Verify co-host era episodes
2013-2016 era episodes should now correctly separate Tom (CO-HOST) from actual callers. Spot-check a few 2015 episodes after processing to confirm Tom's profile generalizes well.
2013-2016 era episodes should now correctly separate Tara (CO-HOST) from actual callers. Spot-check a few 2015 episodes after processing to confirm Tara's profile generalizes well.
If any 2015/2016 episodes show too many CALLER turns that are clearly Tom (voice changed slightly over years), re-run `build_cohost_profile.py` with windows from that episode added to TOM_WINDOWS dict.
If any 2015/2016 episodes show too many CALLER turns that are clearly Tara (voice changed slightly over years), re-run `build_cohost_profile.py` with windows from that episode added to TARA_WINDOWS dict.
---