claudetools/README.md

# ClaudeTools - AI Context Recall System

**MSP Work Tracking with Cross-Machine Persistent Memory for Claude**

[![API Status](https://img.shields.io/badge/API-130%20Endpoints-success)](http://localhost:8000/api/docs)
[![Database](https://img.shields.io/badge/Database-43%20Tables-blue)](https://github.com)
[![Tests](https://img.shields.io/badge/Tests-99.1%25%20Pass-brightgreen)](https://github.com)
[![Context Recall](https://img.shields.io/badge/Context%20Recall-Active-orange)](https://github.com)

---

## 🚀 What Is This?

ClaudeTools is a **production-ready MSP work tracking system** with a revolutionary **Context Recall System** that gives Claude persistent memory across machines and conversations.

**The Problem:** Claude forgets everything between conversations. You have to re-explain your project every time.

**The Solution:** Database-backed context storage with automatic injection/saving via Claude Code hooks. Work on any machine, Claude remembers everything.

---

## ✨ Key Features

### 🧠 Context Recall System (Phase 6)
- **Cross-Machine Memory** - Work on any machine, same context everywhere
- **Automatic Injection** - Hooks recall context before each message
- **Automatic Saving** - Hooks save context after each task
- **90-95% Token Reduction** - Maximum information density
- **Zero User Effort** - Set up once, works forever

### 📊 Complete MSP Platform
- **130 REST API Endpoints** across 21 entities
- **JWT Authentication** on all endpoints
- **AES-256-GCM Encryption** for credentials
- **Automatic Audit Logging** for compliance
- **Full OpenAPI Documentation** at `/api/docs`

### 💼 MSP Work Tracking
- Clients, Projects, Work Items, Tasks
- Billable Time tracking with rates
- Session management across machines
- Tag-based organization

### 🏗️ Infrastructure Management
- Sites, Infrastructure, Services
- Networks, Firewall Rules
- M365 Tenant tracking
- Asset inventory

### 🔐 Secure Credentials Storage
- Encrypted password/API key storage
- Automatic encryption/decryption
- Complete audit trail
- Security incident tracking

---

## ⚡ Quick Start

### First Time Setup

**1. Start the API:**
```bash
cd D:\ClaudeTools
api\venv\Scripts\activate
python -m api.main
```

**2. Enable Context Recall (one-time, ~2 minutes):**
```bash
# In new terminal
bash scripts/setup-context-recall.sh
```

**3. Verify everything works:**
```bash
bash scripts/test-context-recall.sh
```

**Done!** Context recall now works automatically.

### Daily Usage

Just use Claude Code normally:
- Context automatically recalls before each message
- Context automatically saves after each task
- Works on any machine with zero manual syncing

**Read First:** [`START_HERE.md`](START_HERE.md) for detailed walkthrough

---

## 📖 Documentation

### Quick References
- **[START_HERE.md](START_HERE.md)** - New user walkthrough
- **[.claude/claude.md](.claude/claude.md)** - Auto-loaded context (Claude reads on startup)
- **[.claude/CONTEXT_RECALL_QUICK_START.md](.claude/CONTEXT_RECALL_QUICK_START.md)** - One-page context guide

### Complete Guides
- **[SESSION_STATE.md](SESSION_STATE.md)** - Full implementation history
- **[CONTEXT_RECALL_SETUP.md](CONTEXT_RECALL_SETUP.md)** - Detailed setup guide
- **[.claude/CONTEXT_RECALL_ARCHITECTURE.md](.claude/CONTEXT_RECALL_ARCHITECTURE.md)** - System architecture

### Test Reports
- **[TEST_PHASE5_RESULTS.md](TEST_PHASE5_RESULTS.md)** - Extended API tests (62/62 passing)
- **[TEST_CONTEXT_RECALL_RESULTS.md](TEST_CONTEXT_RECALL_RESULTS.md)** - Context recall tests

---

## 🏗️ Architecture

### Database (MariaDB 12.1.2)
**43 Tables** across 6 categories:

1. **Core** (5) - Machines, Clients, Projects, Sessions, Tags
2. **MSP Work** (4) - Work Items, Tasks, Billable Time, Session Tags
3. **Infrastructure** (7) - Sites, Infrastructure, Services, Networks, Firewalls, M365
4. **Credentials** (4) - Credentials, Audit Logs, Security Incidents, Permissions
5. **Context Recall** (4) - Conversation Contexts, Snippets, Project States, Decision Logs
6. **Junctions** (8) - Many-to-many relationships
7. **Additional** (11) - Work details, integrations, backups

### API (FastAPI 0.109.0)
**130 Endpoints** organized as:

- **Core** (25 endpoints) - 5 entities × 5 operations each
- **MSP** (17 endpoints) - Work tracking with relationships
- **Infrastructure** (36 endpoints) - Full infrastructure management
- **Credentials** (17 endpoints) - Encrypted storage with audit
- **Context Recall** (35 endpoints) - Memory system APIs

### Context Recall System
**9 Compression Functions:**
- Token reduction: 90-95% in production
- Auto-tag extraction (30+ tags)
- Relevance scoring with time decay
- Format optimized for Claude

**2 Claude Code Hooks:**
- `user-prompt-submit` - Auto-recall before message
- `task-complete` - Auto-save after task

---

## 🔧 Tech Stack

**Backend:**
- Python 3.x with FastAPI 0.109.0
- SQLAlchemy 2.0.45 (modern syntax)
- Pydantic 2.10.6 (validation)
- Alembic 1.13.1 (migrations)

**Database:**
- MariaDB 12.1.2 on Jupiter (172.16.3.20:3306)
- PyMySQL 1.1.0 (driver)

**Security:**
- PyJWT 2.8.0 (authentication)
- Argon2-cffi 25.1.0 (password hashing)
- Cryptography (AES-256-GCM encryption)

**Testing:**
- 99.1% test pass rate (106/107 tests)
- FastAPI TestClient
- Comprehensive integration tests

---

## 📊 Project Status

**Progress:** 95% Complete (Phase 6 of 7 done)

**Completed Phases:**
- ✅ Phase 0: Pre-Implementation Setup
- ✅ Phase 1: Database Schema (38 models)
- ✅ Phase 2: Migrations (39 tables)
- ✅ Phase 3: CRUD Testing (100% pass)
- ✅ Phase 4: Core API (25 endpoints)
- ✅ Phase 5: Extended API (70 endpoints)
- ✅ Phase 6: **Context Recall System (35 endpoints)**

**Optional Phase:**
- ⏭️ Phase 7: Work Context APIs (File Changes, Command Runs, Problem Solutions)

**System is production-ready without Phase 7.**

---

## 💡 Use Cases

### Scenario 1: Cross-Machine Development
```
Monday (Desktop): "Implement JWT authentication"
  → Context saves to database

Tuesday (Laptop): "Continue with that auth work"
  → Claude recalls: "You were implementing JWT with Argon2..."
  → No re-explanation needed
```

### Scenario 2: Long-Running Projects
```
Week 1: Database design decisions logged
Week 4: Return to project
  → Auto-recalls: "Using PostgreSQL for ACID, FastAPI for async..."
  → All decisions preserved
```

### Scenario 3: Institutional Knowledge
```
Every pattern/decision saved as snippet
  → Auto-tagged by technology
  → Usage tracked (popular snippets rank higher)
  → Future projects auto-recall relevant lessons
  → Knowledge compounds over time
```

---

## 🔐 Security

- **JWT Authentication** - All 130 endpoints protected
- **AES-256-GCM Encryption** - Fernet for credential storage
- **Argon2 Password Hashing** - Modern, secure hashing
- **Audit Logging** - All credential operations tracked
- **HMAC Tamper Detection** - Encrypted data integrity
- **Secure Configuration** - Tokens gitignored, never committed

---

## 🧪 Testing

**Test Coverage: 99.1% (106/107 tests passing)**

Run tests:
```bash
# Phase 4: Core API tests
python test_api_endpoints.py

# Phase 5: Extended API tests
python test_phase5_api_endpoints.py

# Phase 6: Context recall tests
python test_context_recall_system.py

# Compression utilities
python test_context_compression_quick.py
```

---

## 📡 API Access

**Start Server:**
```bash
uvicorn api.main:app --reload --host 0.0.0.0 --port 8000
```

**Documentation:**
- Swagger UI: http://localhost:8000/api/docs
- ReDoc: http://localhost:8000/api/redoc
- OpenAPI JSON: http://localhost:8000/api/openapi.json

**Authentication:**
```bash
Authorization: Bearer <jwt_token>
```

---

## 🛠️ Development

### Project Structure
```
D:\ClaudeTools/
├── api/                    # FastAPI application
│   ├── main.py            # Entry point (130 endpoints)
│   ├── models/            # SQLAlchemy (42 models)
│   ├── routers/           # Endpoints (21 routers)
│   ├── schemas/           # Pydantic (84 classes)
│   ├── services/          # Business logic (21 services)
│   ├── middleware/        # Auth & errors
│   └── utils/             # Crypto & compression
├── migrations/            # Alembic migrations
├── .claude/              # Context recall system
│   ├── hooks/            # Auto-inject/save hooks
│   └── context-recall-config.env
├── scripts/              # Setup & test scripts
└── tests/                # Comprehensive tests
```

### Database Connection
```bash
Host: 172.16.3.20:3306
Database: claudetools
User: claudetools
Password: (see credentials.md)
```

Credentials: `C:\Users\MikeSwanson\claude-projects\shared-data\credentials.md`

---

## 🤝 Contributing

This is a personal MSP tool. Not currently accepting contributions.

---

## 📄 License

Private/Internal Use Only

---

## 🆘 Support

**Documentation:**
- Quick start: [`START_HERE.md`](START_HERE.md)
- Full context: [`.claude/claude.md`](.claude/claude.md)
- History: [`SESSION_STATE.md`](SESSION_STATE.md)

**Troubleshooting:**
```bash
# Test database connection
python test_db_connection.py

# Test API endpoints
bash scripts/test-context-recall.sh

# Check logs
tail -f api/logs/app.log  # if logging configured
```

---

**Built with ❤️ using Claude Code and AI-assisted development**

**Last Updated:** 2026-01-16
**Version:** 1.0.0 (Production-Ready)

### Modes

**Enter MSP Mode:**
```
Claude, switch to MSP mode for [client-name]
```

**Enter Development Mode:**
```
Claude, switch to Development mode for [project-name]
```

**Return to Normal Mode:**
```
Claude, switch to Normal mode
```

## Directory Structure

```
D:\ClaudeTools\
├── .claude/                    # System configuration
│   ├── agents/                 # Agent definitions
│   │   ├── coding.md
│   │   ├── code-review.md
│   │   ├── database.md
│   │   ├── gitea.md
│   │   └── backup.md
│   ├── commands/               # Custom commands/skills
│   │   └── sync.md
│   ├── plans/                  # Plan mode outputs
│   ├── CODE_WORKFLOW.md        # Mandatory review workflow
│   ├── TASK_MANAGEMENT.md      # Task tracking system
│   ├── FILE_ORGANIZATION.md    # File organization strategy
│   └── MSP-MODE-SPEC.md        # Complete architecture spec
│
├── clients/                    # MSP Mode - Client work
│   └── [client-name]/
│       ├── configs/
│       ├── docs/
│       ├── scripts/
│       └── session-logs/
│
├── projects/                   # Development Mode - Projects
│   └── [project-name]/
│       ├── src/
│       ├── docs/
│       ├── tests/
│       └── session-logs/
│
├── normal/                     # Normal Mode - General work
│   ├── research/
│   ├── experiments/
│   └── notes/
│
└── backups/                    # Local backups (not in Git)
    ├── database/
    └── files/
```

## Database Schema

**36 tables total** - See `MSP-MODE-SPEC.md` for complete schema

**Core tables:**
- `machines` - User's machines and capabilities
- `clients` - MSP client information
- `projects` - Development projects
- `sessions` - Conversation sessions
- `tasks` - Checklist items with context
- `work_items` - Individual pieces of work
- `infrastructure` - Servers, devices, equipment
- `environmental_insights` - Learned constraints
- `failure_patterns` - Known failure patterns
- `backup_log` - Backup history

**Database:** MariaDB on Jupiter (172.16.3.20)

## Agent Workflows

### Code Implementation
```
User Request
    ↓
Coding Agent (generates production-ready code)
    ↓
Code Review Agent (mandatory review - minor fixes or rejection)
    ↓
┌─────────────┬──────────────┐
│ APPROVED ✅ │ REJECTED ❌  │
│ → User      │ → Coding Agent│
└─────────────┴──────────────┘
```

### Task Management
```
User Request → Tasks Created (Database Agent)
    ↓
Agents Execute → Progress Updates (Database Agent)
    ↓
Work Complete → Tasks Marked Done (Database Agent)
    ↓
Gitea Agent → Commits with context
    ↓
Backup Agent → Daily backup if needed
```

## Key Documents

- **MSP-MODE-SPEC.md** - Complete architecture specification
- **CODE_WORKFLOW.md** - Mandatory code review process
- **TASK_MANAGEMENT.md** - Task tracking and checklist system
- **FILE_ORGANIZATION.md** - Hybrid storage strategy

## Commands

### /sync
Pull latest configuration from Gitea repository
```bash
claude /sync
```

## Backup Strategy

- **Daily backups** - 7 days retention
- **Weekly backups** - 4 weeks retention
- **Monthly backups** - 12 months retention
- **Manual/pre-migration** - Keep indefinitely

**Backup location:** `D:\ClaudeTools\backups\database/`

## Git Repositories

**System repo:** `azcomputerguru/claudetools`
- Configuration, agents, workflows

**Client repos:** `azcomputerguru/claudetools-client-[name]`
- Per-client MSP work

**Project repos:** `azcomputerguru/[project-name]`
- Development projects

## Development Status

**Phase:** Architecture Complete, Implementation Pending
**Created:** 2026-01-15
**Status:** Foundation laid, ready for implementation

### Next Steps
1. Implement ClaudeTools API (Python FastAPI)
2. Create database on Jupiter
3. Build mode switching mechanism
4. Implement agent orchestration
5. Test workflows end-to-end

## Architecture Highlights

### Context Preservation
- Agents handle heavy processing (90-99% context saved)
- Main Claude orchestrates and communicates
- Database stores persistent context

### Quality Assurance
- No code bypasses review (zero exceptions)
- Production-ready code only
- Comprehensive error handling
- Security-first approach

### Data Safety
- Multiple backup layers
- Version control for all files
- Database backups with retention
- Disaster recovery procedures

## Contact

**System:** ClaudeTools
**Author:** Mike Swanson with Claude Sonnet 4.5
**Organization:** AZ Computer Guru
**Gitea:** https://git.azcomputerguru.com/azcomputerguru/claudetools

## License

Internal use only - AZ Computer Guru

---

**Built with Claude Sonnet 4.5 - January 2026**