claudetools/GREPAI_SYNC_STRATEGY.md

Grepai Sync Strategy

Purpose: Keep grepai indexes synchronized between Windows and Mac development machines

---

Understanding Grepai Index

What is the index?

• Semantic embeddings of your codebase (13,020 chunks from 961 files)
• Size: 73.7 MB
• Generated using: nomic-embed-text model via Ollama
• Stored locally: .grepai/ directory (usually)

Index components:

• Embeddings database (vector representations of code)
• Symbol tracking database (functions, classes, etc.)
• File metadata (paths, timestamps, hashes)

---

Sync Strategy Options

Option 1: Independent Indexes (RECOMMENDED)

How it works:

• Each machine maintains its own grepai index
• Index is gitignored (not committed to repository)
• Each machine rebuilds index from local codebase

Advantages:

• [OK] Always consistent with local codebase
• [OK] No merge conflicts
• [OK] Handles machine-specific paths correctly
• [OK] Simple and reliable

Disadvantages:

• [WARNING] Must rebuild index on each machine (one-time setup)
• [WARNING] Initial indexing takes time (~2-5 minutes for 961 files)

Setup:

# Add to .gitignore
echo ".grepai/" >> .gitignore

# On each machine:
grepai init
grepai index

# Keep codebase in sync via git
git pull origin main
grepai index  # Rebuild after pulling changes

When to rebuild:

• After pulling major code changes (>50 files)
• After switching branches
• If search results seem outdated
• Weekly maintenance (optional)

---

Option 2: Shared Index via Git

How it works:

• Commit .grepai/ directory to repository
• Pull index along with code changes

Advantages:

• [OK] Instant sync (no rebuild needed)
• [OK] Same index on all machines

Disadvantages:

• [ERROR] Can cause merge conflicts
• [ERROR] May have absolute path issues (D:\ vs ~/)
• [ERROR] Index may get out of sync with actual code
• [ERROR] Increases repository size (+73.7 MB)

NOT RECOMMENDED due to path conflicts and sync issues.

---

Option 3: Automated Rebuild on Pull (BEST PRACTICE)

How it works:

• Keep indexes independent (Option 1)
• Automatically rebuild index after git pull
• Use git hooks to trigger rebuild

Setup:

Create .git/hooks/post-merge (git pull trigger):

#!/bin/bash
echo "[grepai] Rebuilding index after merge..."
grepai index --quiet
echo "[OK] Index updated"

Make executable:

chmod +x .git/hooks/post-merge

Advantages:

• [OK] Always up to date
• [OK] Automated (no manual intervention)
• [OK] No merge conflicts
• [OK] Each machine has correct index

Disadvantages:

• [WARNING] Adds 1-2 minutes to git pull time
• [WARNING] Requires git hook setup on each machine

---

Recommended Workflow

Initial Setup (One-Time Per Machine)

On Windows:

# Ensure .grepai is gitignored
echo ".grepai/" >> .gitignore
git add .gitignore
git commit -m "chore: gitignore grepai index"

# Build index
grepai index

On Mac:

# Pull latest code
git pull origin main

# Install Ollama models
ollama pull nomic-embed-text

# Build index
grepai index

Daily Workflow

Start of day (on either machine):

# Update codebase
git pull origin main

# Rebuild index (if significant changes)
grepai index

During development:

• No action needed
• Grepai auto-updates as you edit files (depending on configuration)

End of day:

# Commit your changes
git add .
git commit -m "your message"
git push origin main

On other machine:

# Pull changes
git pull origin main

# Rebuild index
grepai index

---

Quick Rebuild Commands

Full rebuild:

grepai index

Incremental update (faster, if supported):

grepai index --incremental

Check if rebuild needed:

# Compare last index time with last git pull
grepai status
git log -1 --format="%ai"

---

Automation Script

Create sync-and-index.sh:

#!/bin/bash
# Sync codebase and rebuild grepai index

echo "=== Syncing ClaudeTools ==="

# Pull latest changes
echo "[1/3] Pulling from git..."
git pull origin main

if [ $? -ne 0 ]; then
    echo "[ERROR] Git pull failed"
    exit 1
fi

# Check if significant changes
CHANGED_FILES=$(git diff HEAD@{1} --name-only | wc -l)
echo "[2/3] Changed files: $CHANGED_FILES"

# Rebuild index if changes detected
if [ "$CHANGED_FILES" -gt 0 ]; then
    echo "[3/3] Rebuilding grepai index..."
    grepai index
    echo "[OK] Sync complete with index rebuild"
else
    echo "[3/3] No changes, skipping index rebuild"
    echo "[OK] Sync complete"
fi

Usage:

chmod +x sync-and-index.sh
./sync-and-index.sh

---

Monitoring Index Health

Check index status:

grepai status

Expected output (healthy):

Total files: 961
Total chunks: 13,020
Index size: 73.7 MB
Last updated: [recent timestamp]
Provider: ollama
Model: nomic-embed-text
Symbols: Ready

Signs of unhealthy index:

• File count doesn't match codebase
• Last updated > 7 days old
• Symbol tracking not ready
• Search results seem wrong

Fix:

grepai rebuild  # or
grepai index --force

---

Best Practices

1. Always gitignore .grepai/ - Prevents merge conflicts
2. Rebuild after major pulls - Keeps index accurate
3. Use same embedding model - Ensures consistency (nomic-embed-text)
4. Verify index health weekly - Run grepai status
5. Document rebuild frequency - Set team expectations

---

Troubleshooting

Index out of sync

# Force complete rebuild
rm -rf .grepai
grepai init
grepai index

Different results on different machines

• Check embedding model: grepai status | grep model
• Should both use: nomic-embed-text
• Rebuild with same model if different

Index too large

# Check what's being indexed
grepai stats

# Add exclusions to .grepai.yml (if exists)
# exclude:
#   - node_modules/
#   - venv/
#   - .git/

---

Summary

RECOMMENDED APPROACH: Option 3 (Automated Rebuild)

Setup:

1. Gitignore .grepai/ directory
2. Install git hook for post-merge rebuild
3. Each machine maintains independent index
4. Index rebuilds automatically after git pull

Maintenance:

• Initial index build: 2-5 minutes (one-time per machine)
• Incremental rebuilds: 30-60 seconds (after pulls)
• Full rebuilds: As needed (weekly or when issues arise)

Key principle: Treat grepai index like compiled artifacts - gitignore them and rebuild from source (the codebase) as needed.

---

Last Updated

2026-01-22 - Initial creation