claudetools/CONTEXT_SAVE_CRITICAL_BUGS.md

Context Save System - Critical Bug Analysis

Date: 2026-01-17 Severity: CRITICAL - Context recall completely non-functional Status: All bugs identified, fixes required

---

Executive Summary

The context save/recall system has 7 CRITICAL BUGS preventing it from working:

1. Encoding Issue (CRITICAL) - Windows cp1252 vs UTF-8 mismatch
2. API Payload Format - Tags field double-serialized as JSON string
3. Missing project_id - Contexts saved without project_id can't be recalled
4. Silent Failure - Errors logged but not visible to user
5. Response Logging - Unicode in API responses crashes logger
6. Active Time Counter Bug - Counter never resets properly
7. No Validation - API accepts malformed payloads without error

---

Bug #1: Windows Encoding Issue (CRITICAL)

File: D:\ClaudeTools\.claude\hooks\periodic_context_save.py (line 42-47) File: D:\ClaudeTools\.claude\hooks\periodic_save_check.py (line 39-43)

Problem:

# Current code (BROKEN)
def log(message):
    timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
    log_message = f"[{timestamp}] {message}\n"

    with open(LOG_FILE, "a", encoding="utf-8") as f:  # File uses UTF-8
        f.write(log_message)

    print(log_message.strip(), file=sys.stderr)  # stderr uses cp1252!

Root Cause:

• File writes with UTF-8 encoding (correct)
• sys.stderr uses cp1252 on Windows (default)
• When API response contains Unicode characters ('\u2717' = ✗), print() crashes
• Log file shows: 'charmap' codec can't encode character '\u2717' in position 22

Evidence:

[2026-01-17 12:01:54] 300s of active time reached - saving context
[2026-01-17 12:01:54] Error in monitor loop: 'charmap' codec can't encode character '\u2717' in position 22: character maps to <undefined>

Fix Required:

def log(message):
    """Write log message to file and stderr with proper encoding"""
    timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
    log_message = f"[{timestamp}] {message}\n"

    # Write to log file with UTF-8 encoding
    with open(LOG_FILE, "a", encoding="utf-8") as f:
        f.write(log_message)

    # Print to stderr with safe encoding (replace unmappable chars)
    try:
        print(log_message.strip(), file=sys.stderr)
    except UnicodeEncodeError:
        # Fallback: encode as UTF-8 bytes, replace unmappable chars
        safe_message = log_message.encode('utf-8', errors='replace').decode('utf-8')
        print(safe_message.strip(), file=sys.stderr)

Alternative Fix (Better): Set PYTHONIOENCODING environment variable at script start:

# At top of script, before any imports
import sys
import os
os.environ['PYTHONIOENCODING'] = 'utf-8'

---

Bug #2: Tags Field Double-Serialization

File: D:\ClaudeTools\.claude\hooks\periodic_context_save.py (line 176) File: D:\ClaudeTools\.claude\hooks\periodic_save_check.py (line 204)

Problem:

# Current code (WRONG)
payload = {
    "context_type": "session_summary",
    "title": title,
    "dense_summary": summary,
    "relevance_score": 5.0,
    "tags": json.dumps(["auto-save", "periodic", "active-session"]),  # WRONG!
}

# requests.post(url, json=payload, headers=headers)
# This double-serializes tags!

What Happens:

1. json.dumps(["auto-save", "periodic"]) → '["auto-save", "periodic"]' (string)
2. requests.post(..., json=payload) → serializes entire payload
3. API receives: {"tags": "\"[\\\"auto-save\\\", \\\"periodic\\\"]\""} (double-escaped!)
4. Database stores: "[\"auto-save\", \"periodic\"]" (escaped string, not JSON array)

Expected vs Actual:

Expected in database:

{"tags": "[\"auto-save\", \"periodic\"]"}

Actual in database (double-serialized):

{"tags": "\"[\\\"auto-save\\\", \\\"periodic\\\"]\""}

Fix Required:

# CORRECT - Let requests serialize it
payload = {
    "context_type": "session_summary",
    "title": title,
    "dense_summary": summary,
    "relevance_score": 5.0,
    "tags": json.dumps(["auto-save", "periodic", "active-session"]),  # Keep as-is
}

# requests.post() will serialize the whole payload correctly

Wait, actually checking the API...

Looking at the schema (api/schemas/conversation_context.py line 25):

tags: Optional[str] = Field(None, description="JSON array of tags for retrieval and categorization")

The field is STRING type, expecting a JSON string! So the current code is CORRECT.

But there's still a bug:

The API response shows tags stored as string:

{"tags": "[\"test\"]"}

But the get_recall_context function (line 204 in service) does:

tags = json.loads(ctx.tags) if ctx.tags else []

So it expects the field to contain a JSON string, which is correct.

Conclusion: Tags serialization is CORRECT. Not a bug.

---

Bug #3: Missing project_id in Payload

File: D:\ClaudeTools\.claude\hooks\periodic_context_save.py (line 162-177) File: D:\ClaudeTools\.claude\hooks\periodic_save_check.py (line 190-205)

Problem:

# Current code (INCOMPLETE)
payload = {
    "context_type": "session_summary",
    "title": title,
    "dense_summary": summary,
    "relevance_score": 5.0,
    "tags": json.dumps(["auto-save", "periodic", "active-session"]),
}
# Missing: project_id!

Impact:

• Context is saved without project_id
• user-prompt-submit hook filters by project_id (line 74 in user-prompt-submit)
• Contexts without project_id are NEVER recalled
• This is why context recall isn't working!

Evidence: Looking at the API response from the test:

{
  "project_id": null,  // <-- BUG! Should be "c3d9f1c8-dc2b-499f-a228-3a53fa950e7b"
  "context_type": "session_summary",
  ...
}

The config file has:

CLAUDE_PROJECT_ID=c3d9f1c8-dc2b-499f-a228-3a53fa950e7b

But the periodic save scripts call detect_project_id() which returns "unknown" if git commands fail.

Fix Required:

def save_periodic_context(config, project_id):
    """Save context to database via API"""
    if not config["jwt_token"]:
        log("No JWT token - cannot save context")
        return False

    # Ensure we have a valid project_id
    if not project_id or project_id == "unknown":
        log("[WARNING] No project_id detected - context may not be recalled")
        # Try to get from config
        project_id = config.get("project_id")

    title = f"Periodic Save - {datetime.now().strftime('%Y-%m-%d %H:%M')}"
    summary = f"Auto-saved context after 5 minutes of active work. Session in progress on project: {project_id}"

    payload = {
        "project_id": project_id,  # ADD THIS!
        "context_type": "session_summary",
        "title": title,
        "dense_summary": summary,
        "relevance_score": 5.0,
        "tags": json.dumps(["auto-save", "periodic", "active-session", project_id]),
    }

Also update load_config():

def load_config():
    """Load configuration from context-recall-config.env"""
    config = {
        "api_url": "http://172.16.3.30:8001",
        "jwt_token": None,
        "project_id": None,  # ADD THIS!
    }

    if CONFIG_FILE.exists():
        with open(CONFIG_FILE) as f:
            for line in f:
                line = line.strip()
                if line.startswith("CLAUDE_API_URL="):
                    config["api_url"] = line.split("=", 1)[1]
                elif line.startswith("JWT_TOKEN="):
                    config["jwt_token"] = line.split("=", 1)[1]
                elif line.startswith("CLAUDE_PROJECT_ID="):  # ADD THIS!
                    config["project_id"] = line.split("=", 1)[1]

    return config

---

Bug #4: Silent Failure - No User Feedback

File: D:\ClaudeTools\.claude\hooks\periodic_context_save.py (line 188-197) File: D:\ClaudeTools\.claude\hooks\periodic_save_check.py (line 215-226)

Problem:

# Current code (SILENT FAILURE)
if response.status_code in [200, 201]:
    log(f"[OK] Context saved successfully (ID: {response.json().get('id', 'unknown')})")
    return True
else:
    log(f"[ERROR] Failed to save context: HTTP {response.status_code}")
    return False

Issues:

1. Errors are only logged to file - user never sees them
2. No details about WHAT went wrong
3. No retry mechanism
4. No notification to user

Fix Required:

if response.status_code in [200, 201]:
    context_id = response.json().get('id', 'unknown')
    log(f"[OK] Context saved (ID: {context_id})")
    return True
else:
    # Log full error details
    error_detail = response.text[:500] if response.text else "No error message"
    log(f"[ERROR] Failed to save context: HTTP {response.status_code}")
    log(f"[ERROR] Response: {error_detail}")

    # Try to parse error details
    try:
        error_json = response.json()
        if "detail" in error_json:
            log(f"[ERROR] Detail: {error_json['detail']}")
    except:
        pass

    return False

---

Bug #5: Unicode in API Response Crashes Logger

File: periodic_context_save.py (line 189)

Problem: When API returns a successful response with Unicode characters, the logger tries to print it and crashes:

log(f"[OK] Context saved successfully (ID: {response.json().get('id', 'unknown')})")

If response.json() contains fields with Unicode (from title, dense_summary, etc.), this crashes when logging to stderr.

Fix Required: Use the encoding-safe log function from Bug #1.

---

Bug #6: Active Time Counter Never Resets

File: periodic_context_save.py (line 223)

Problem:

# Check if we've reached the save interval
if state["active_seconds"] >= SAVE_INTERVAL_SECONDS:
    log(f"{SAVE_INTERVAL_SECONDS}s of active time reached - saving context")

    project_id = detect_project_id()
    if save_periodic_context(config, project_id):
        state["last_save"] = datetime.now(timezone.utc).isoformat()

    # Reset timer
    state["active_seconds"] = 0
    save_state(state)

Issue: Look at the log:

[2026-01-17 12:01:54] Active: 300s / 300s
[2026-01-17 12:01:54] 300s of active time reached - saving context
[2026-01-17 12:01:54] Error in monitor loop: 'charmap' codec can't encode character '\u2717'
[2026-01-17 12:02:55] Active: 360s / 300s   <-- Should be 60s, not 360s!

The counter is NOT resetting because the exception is caught by the outer try/except at line 243:

except Exception as e:
    log(f"Error in monitor loop: {e}")
    time.sleep(CHECK_INTERVAL_SECONDS)

When save_periodic_context() throws an encoding exception, it's caught, logged, and execution continues WITHOUT resetting the counter.

Fix Required:

# Check if we've reached the save interval
if state["active_seconds"] >= SAVE_INTERVAL_SECONDS:
    log(f"{SAVE_INTERVAL_SECONDS}s of active time reached - saving context")

    project_id = detect_project_id()

    # Always reset timer, even if save fails
    save_success = False
    try:
        save_success = save_periodic_context(config, project_id)
        if save_success:
            state["last_save"] = datetime.now(timezone.utc).isoformat()
    except Exception as e:
        log(f"[ERROR] Exception during save: {e}")
    finally:
        # Always reset timer to prevent repeated attempts
        state["active_seconds"] = 0
        save_state(state)

---

Bug #7: No API Payload Validation

File: All periodic save scripts

Problem: The scripts don't validate the payload before sending to API:

• No check if JWT token is valid/expired
• No check if project_id is a valid UUID
• No check if API is reachable before building payload

Fix Required:

def save_periodic_context(config, project_id):
    """Save context to database via API"""
    # Validate JWT token exists
    if not config.get("jwt_token"):
        log("[ERROR] No JWT token - cannot save context")
        return False

    # Validate project_id
    if not project_id or project_id == "unknown":
        log("[WARNING] No valid project_id - trying config")
        project_id = config.get("project_id")
        if not project_id:
            log("[ERROR] No project_id available - context won't be recallable")
            # Continue anyway, but log warning

    # Validate project_id is UUID format
    try:
        import uuid
        uuid.UUID(project_id)
    except (ValueError, AttributeError):
        log(f"[ERROR] Invalid project_id format: {project_id}")
        # Continue with string ID anyway

    # Rest of function...

---

Additional Issues Found

Issue A: Database Connection Test Shows "Not authenticated"

The API at http://172.16.3.30:8001 is running (returns HTML on /api/docs), but direct context fetch returns:

{"detail":"Not authenticated"}

Wait, that was WITHOUT the auth header. WITH the auth header:

{
  "total": 118,
  "contexts": [...]
}

So the API IS working. Not a bug.

---

Issue B: Context Recall Hook Not Injecting

File: user-prompt-submit (line 79-94)

The hook successfully retrieves contexts from API:

CONTEXT_RESPONSE=$(curl -s --max-time 3 \
    "${RECALL_URL}?${QUERY_PARAMS}" \
    -H "Authorization: Bearer ${JWT_TOKEN}" \
    -H "Accept: application/json" 2>/dev/null)

But the issue is: contexts don't have matching project_id, so the query returns empty.

Query URL:

http://172.16.3.30:8001/api/conversation-contexts/recall?project_id=c3d9f1c8-dc2b-499f-a228-3a53fa950e7b&limit=10&min_relevance_score=5.0

Database contexts have:

{"project_id": null}  // <-- Won't match!

Root Cause: Bug #3 (missing project_id in payload)

---

Summary of Required Fixes

Priority 1 (CRITICAL - Blocking all functionality):

1. Fix encoding issue in periodic save scripts (Bug #1)

   • Add PYTHONIOENCODING environment variable
   • Use safe stderr printing

2. Add project_id to payload in periodic save scripts (Bug #3)

   • Load project_id from config
   • Include in API payload
   • Validate UUID format

3. Fix active time counter in periodic save daemon (Bug #6)

   • Always reset counter in finally block
   • Prevent repeated save attempts

Priority 2 (Important - Better error handling):

4. Improve error logging (Bug #4)

   • Log full API error responses
   • Show detailed error messages
   • Add retry mechanism

5. Add payload validation (Bug #7)

   • Validate JWT token exists
   • Validate project_id format
   • Check API reachability

Priority 3 (Nice to have):

6. Add user notifications
   • Show context save success/failure in Claude UI
   • Alert when context recall fails
   • Display periodic save status

---

Files Requiring Changes

1. D:\ClaudeTools\.claude\hooks\periodic_context_save.py

   • Lines 1-5: Add PYTHONIOENCODING
   • Lines 37-47: Fix log() function encoding
   • Lines 50-66: Add project_id to config loading
   • Lines 162-197: Add project_id to payload, improve error handling
   • Lines 223-232: Fix active time counter reset

2. D:\ClaudeTools\.claude\hooks\periodic_save_check.py

   • Lines 1-5: Add PYTHONIOENCODING
   • Lines 34-43: Fix log() function encoding
   • Lines 46-62: Add project_id to config loading
   • Lines 190-226: Add project_id to payload, improve error handling

3. D:\ClaudeTools\.claude\hooks\task-complete

   • Lines 79-115: Should already include project_id (verify)

4. D:\ClaudeTools\.claude\context-recall-config.env

   • Already has CLAUDE_PROJECT_ID (no changes needed)

---

Testing Checklist

After fixes are applied:

• Periodic save runs without encoding errors
• Contexts are saved with correct project_id
• Active time counter resets properly
• Context recall hook retrieves saved contexts
• API errors are logged with full details
• Invalid project_ids are handled gracefully
• JWT token expiration is detected
• Unicode characters in titles/summaries work correctly

---

Root Cause Analysis

Why did this happen?

1. Encoding issue: Developed on Unix/Mac (UTF-8 everywhere), deployed on Windows (cp1252 default)
2. Missing project_id: Tested with manual API calls (included project_id), but periodic saves used auto-detection (failed silently)
3. Counter bug: Exception handling too broad, caught save failures without cleanup
4. Silent failures: Background daemon has no user-visible output

Prevention:

1. Test on Windows with cp1252 encoding
2. Add integration tests that verify end-to-end flow
3. Add health check endpoint that validates configuration
4. Add user-visible status indicators for context saves

---

Generated: 2026-01-17 15:45 PST Total Bugs Found: 7 (3 Critical, 2 Important, 2 Nice-to-have) Status: Analysis complete, fixes ready to implement