claudetools/.claude/DATABASE_FIRST_PROTOCOL.md

Database-First Protocol

CRITICAL: This protocol MUST be followed for EVERY user request.

---

The Problem

Currently, Claude:

1. Receives user request
2. Searches local files (maybe)
3. Performs work
4. (Never saves context automatically)

This wastes tokens, misses critical context, and loses work across sessions.

---

The Solution: Database-First Protocol

MANDATORY FIRST STEP - For EVERY User Request

BEFORE doing ANYTHING else:

1. Query the context database for relevant information
2. Inject retrieved context into your working memory
3. THEN proceed with the user's request

---

Implementation

Step 1: Check Database (ALWAYS FIRST)

Before analyzing the user's request, run this command:

curl -s -H "Authorization: Bearer $JWT_TOKEN" \
  "http://172.16.3.30:8001/api/conversation-contexts/recall?\
search_term={user_keywords}&limit=10" | python -m json.tool

Extract keywords from user request. Examples:

• User: "What's the status of Dataforth project?" → search_term=dataforth
• User: "Continue work on GuruConnect" → search_term=guruconnect
• User: "Fix the API bug" → search_term=API+bug
• User: "Help with database" → search_term=database

Step 2: Review Retrieved Context

The API returns up to 10 relevant contexts with:

• title - Short description
• dense_summary - Compressed context (90% token reduction)
• relevance_score - How relevant (0-10)
• tags - Keywords for filtering
• created_at - Timestamp

Step 3: Use Context in Your Response

Reference the context when responding:

• "Based on previous context from {date}..."
• "According to the database, Dataforth DOS project..."
• "Context shows this was last discussed on..."

Step 4: Save New Context (After Completion)

After completing a significant task:

curl -s -H "Authorization: Bearer $JWT_TOKEN" \
  -X POST "http://172.16.3.30:8001/api/conversation-contexts" \
  -H "Content-Type: application/json" \
  -d '{
    "project_id": "c3d9f1c8-dc2b-499f-a228-3a53fa950e7b",
    "context_type": "session_summary",
    "title": "Brief title of what was accomplished",
    "dense_summary": "Compressed summary of work done, decisions made, files changed",
    "relevance_score": 7.0,
    "tags": "[\"keyword1\", \"keyword2\", \"keyword3\"]"
  }'

---

When to Save Context

Save context automatically when:

1. Task Completion - TodoWrite task marked as completed
2. Major Decision - Architectural choice, approach selection
3. File Changes - Significant code changes (>50 lines)
4. Problem Solved - Bug fixed, issue resolved
5. User Requests - Via /snapshot command
6. Session End - Before closing conversation

---

Agent Delegation Rules

Main Claude is a COORDINATOR, not an EXECUTOR.

Before performing any task, check delegation table:

Task Type	Delegate To	Always?
Context retrieval	Database Agent	[OK] YES
Codebase search	Explore Agent	For patterns/keywords
Code changes >10 lines	Coding Agent	[OK] YES
Running tests	Testing Agent	[OK] YES
Git operations	Gitea Agent	[OK] YES
File operations <5 files	Main Claude	Direct OK
Documentation	Documentation Squire	For comprehensive docs

How to Delegate:

Instead of: Searching files directly with Grep/Glob
Do: "Let me delegate to the Explore agent to search the codebase..."

Instead of: Writing code directly
Do: "Let me delegate to the Coding Agent to implement this change..."

Instead of: Running tests yourself
Do: "Let me delegate to the Testing Agent to run the test suite..."

---

Context Database Quick Reference

Query Endpoints

# Search by term
GET /api/conversation-contexts/recall?search_term={term}&limit=10

# Filter by tags
GET /api/conversation-contexts/recall?tags=dataforth&tags=dos&limit=10

# Get by project
GET /api/conversation-contexts/recall?project_id={uuid}&limit=10

# List all recent
GET /api/conversation-contexts?limit=50

Save Endpoint

POST /api/conversation-contexts
{
  "project_id": "uuid",
  "context_type": "session_summary|checkpoint|decision|problem_solution",
  "title": "Short title",
  "dense_summary": "Compressed summary with key info",
  "relevance_score": 1.0-10.0,
  "tags": "[\"tag1\", \"tag2\"]"
}

---

Example Workflow

User Request: "What's the status of the Dataforth DOS project?"

WRONG Approach:

Claude: Let me search local files...
(Wastes tokens, misses imported context in database)

CORRECT Approach:

Claude: Let me check the context database first...

[Runs: curl .../recall?search_term=dataforth]

Claude: "Based on context retrieved from the database, the Dataforth
DOS machines project involves analyzing drive images from test machines
with ATE (Automated Test Equipment) software. The conversation was
imported on 2026-01-18 and includes 1,241KB of data.

The project appears to focus on Dataforth industrial I/O equipment
testing (5B, 7B, 8B series modules).

Would you like me to delegate to the Explore agent to find specific
files related to this project?"

---

Integration with Hooks

The hooks in .claude/hooks/ should assist but NOT replace manual queries:

• user-prompt-submit - Auto-injects context (passive)
• task-complete - Auto-saves context (passive)

BUT: You should ACTIVELY query database yourself before major work.

Don't rely solely on hooks. They're a backup, not the primary mechanism.

---

Token Efficiency

Before Database-First:

• Read 3MB of local files: ~750,000 tokens
• Parse conversation histories: ~250,000 tokens
• Total: ~1,000,000 tokens per session

After Database-First:

• Query database: 500 tokens (API call)
• Receive compressed summaries: ~5,000 tokens (10 contexts)
• Total: ~5,500 tokens per session

Savings: 99.4% token reduction

---

Troubleshooting

Database Query Returns Empty

# Check if API is up
curl http://172.16.3.30:8001/health

# Check total contexts
curl -H "Authorization: Bearer $JWT" \
  http://172.16.3.30:8001/api/conversation-contexts | \
  python -c "import sys,json; print(f'Total: {json.load(sys.stdin)[\"total\"]}')"

# Try different search term
# Instead of: search_term=dataforth%20DOS
# Try: search_term=dataforth

Authentication Fails

# Check JWT token in config
cat .claude/context-recall-config.env | grep JWT_TOKEN

# Verify token not expired
# Current token expires: 2026-02-16

No Results for Known Project

The recall endpoint uses PostgreSQL full-text search. Try:

• Simpler search terms
• Individual keywords instead of phrases
• Checking tags directly: ?tags=dataforth

---

Enforcement

This protocol is MANDATORY. To ensure compliance:

1. Every response should start with "Checking database for context..."
2. Before major work, always query database
3. After completion, always save summary
4. For delegation, use agents not direct execution

Violation Example:

User: "Find all Python files"
Claude: [Runs Glob directly] [ERROR] WRONG

Correct:
Claude: "Let me delegate to Explore agent to search for Python files" [OK]

---

Last Updated: 2026-01-18 Status: ACTIVE - MUST BE FOLLOWED Priority: CRITICAL