feat: Major directory reorganization and cleanup
Reorganized project structure for better maintainability and reduced disk usage by 95.9% (11 GB -> 451 MB). Directory Reorganization (85% reduction in root files): - Created docs/ with subdirectories (deployment, testing, database, etc.) - Created infrastructure/vpn-configs/ for VPN scripts - Moved 90+ files from root to organized locations - Archived obsolete documentation (context system, offline mode, zombie debugging) - Moved all test files to tests/ directory - Root directory: 119 files -> 18 files Disk Cleanup (10.55 GB recovered): - Deleted Rust build artifacts: 9.6 GB (target/ directories) - Deleted Python virtual environments: 161 MB (venv/ directories) - Deleted Python cache: 50 KB (__pycache__/) New Structure: - docs/ - All documentation organized by category - docs/archives/ - Obsolete but preserved documentation - infrastructure/ - VPN configs and SSH setup - tests/ - All test files consolidated - logs/ - Ready for future logs Benefits: - Cleaner root directory (18 vs 119 files) - Logical organization of documentation - 95.9% disk space reduction - Faster navigation and discovery - Better portability (build artifacts excluded) Build artifacts can be regenerated: - Rust: cargo build --release (5-15 min per project) - Python: pip install -r requirements.txt (2-3 min) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
186
docs/archives/agent-deliveries/AGENT4_DELIVERY.md
Normal file
186
docs/archives/agent-deliveries/AGENT4_DELIVERY.md
Normal file
@@ -0,0 +1,186 @@
|
||||
# Coding Agent #4 - Wave 2 Delivery Report
|
||||
|
||||
**Agent:** Coding Agent #4
|
||||
**Assignment:** Context Learning + Integrations + Backup + API + Junction (12 models)
|
||||
**Date:** 2026-01-15
|
||||
**Status:** Partially Complete (7 of 12 models created)
|
||||
|
||||
---
|
||||
|
||||
## Models Created (7 models)
|
||||
|
||||
### Context Learning (1 model)
|
||||
1. **environmental_insight.py** ✅ - `environmental_insights` table
|
||||
- Stores learned insights about client/infrastructure environments
|
||||
- Categories: command_constraints, service_configuration, version_limitations, etc.
|
||||
- Confidence levels: confirmed, likely, suspected
|
||||
- Priority system (1-10) for insight importance
|
||||
|
||||
### Integrations (3 models)
|
||||
2. **external_integration.py** ✅ - `external_integrations` table
|
||||
- Logs all interactions with external systems (SyncroMSP, MSP Backups, Zapier)
|
||||
- Tracks request/response data as JSON
|
||||
- Direction tracking (inbound/outbound)
|
||||
- Action tracking (created, updated, linked, attached)
|
||||
|
||||
3. **integration_credential.py** ✅ - `integration_credentials` table
|
||||
- Stores encrypted OAuth tokens, API keys, and credentials
|
||||
- Supports oauth, api_key, and basic_auth credential types
|
||||
- All sensitive data encrypted with AES-256-GCM (stored as BYTEA/LargeBinary)
|
||||
- Connection testing status tracking
|
||||
|
||||
4. **ticket_link.py** ✅ - `ticket_links` table
|
||||
- Links ClaudeTools sessions to external ticketing systems
|
||||
- Supports SyncroMSP, Autotask, ConnectWise
|
||||
- Link types: related, resolves, documents
|
||||
- Tracks ticket status and URLs
|
||||
|
||||
### Backup (1 model)
|
||||
5. **backup_log.py** ✅ - `backup_log` table
|
||||
- Tracks all ClaudeTools database backups
|
||||
- Backup types: daily, weekly, monthly, manual, pre-migration
|
||||
- Verification status: passed, failed, not_verified
|
||||
- Duration calculation in application layer (not stored generated column)
|
||||
- Default backup method: mysqldump
|
||||
|
||||
### Junction Tables (2 models)
|
||||
6. **work_item_tag.py** ✅ - `work_item_tags` junction table
|
||||
- Many-to-many: work_items ↔ tags
|
||||
- Composite primary key (work_item_id, tag_id)
|
||||
- CASCADE delete on both sides
|
||||
|
||||
7. **infrastructure_tag.py** ✅ - `infrastructure_tags` junction table
|
||||
- Many-to-many: infrastructure ↔ tags
|
||||
- Composite primary key (infrastructure_id, tag_id)
|
||||
- CASCADE delete on both sides
|
||||
- **Note:** Not explicitly in spec, but inferred from pattern and mentioned in line 1548
|
||||
|
||||
---
|
||||
|
||||
## Models NOT Created (5 models) - Not Found in Spec
|
||||
|
||||
The following tables from the assignment were NOT found in MSP-MODE-SPEC.md:
|
||||
|
||||
### Context Learning (2 missing)
|
||||
- **environmental_examples** - No table definition found
|
||||
- **learning_metrics** - No table definition found
|
||||
|
||||
### Backup (1 missing)
|
||||
- **backup_schedules** - No table definition found
|
||||
- Note: `backup_log` exists for tracking completed backups
|
||||
- A schedules table would be for planning future backups
|
||||
|
||||
### API Users (2 missing)
|
||||
- **api_users** - No table definition found
|
||||
- **api_tokens** - No table definition found
|
||||
- Note: The spec mentions JWT tokens in INITIAL_DATA.md but no dedicated user/token tables
|
||||
|
||||
---
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
### Design Decisions
|
||||
|
||||
1. **Computed Columns**: The `backup_log.duration_seconds` field is NOT a stored generated column (TIMESTAMPDIFF not portable). Instead, a helper method `calculate_duration()` computes it in Python.
|
||||
|
||||
2. **Encryption**: `integration_credentials` uses `LargeBinary` (SQLAlchemy) which maps to BYTEA (PostgreSQL) or BLOB (MySQL/MariaDB) for encrypted credential storage.
|
||||
|
||||
3. **Timestamps**: Models use `TimestampMixin` where appropriate, except junction tables which don't need timestamps.
|
||||
|
||||
4. **Foreign Keys**: All use `CHAR(36)` for UUID compatibility with MariaDB.
|
||||
|
||||
5. **Infrastructure Tags**: Created based on inference from spec mentions and pattern consistency with other junction tables.
|
||||
|
||||
### SQLAlchemy 2.0 Patterns Used
|
||||
|
||||
- ✅ `Mapped[type]` annotations
|
||||
- ✅ `mapped_column()` for all columns
|
||||
- ✅ Proper type hints with `Optional[]`
|
||||
- ✅ `CheckConstraint` for enum-like values
|
||||
- ✅ `Index()` in `__table_args__`
|
||||
- ✅ Relationship comments (not activated to avoid circular imports)
|
||||
- ✅ `__repr__()` methods for debugging
|
||||
|
||||
### Indexes Created
|
||||
|
||||
All models have proper indexes matching the spec:
|
||||
- `environmental_insights`: client, infrastructure, category
|
||||
- `external_integrations`: session, type, external_id
|
||||
- `integration_credentials`: integration_name
|
||||
- `ticket_links`: session, client, (integration_type, ticket_id) composite
|
||||
- `backup_log`: backup_type, backup_completed_at, verification_status
|
||||
- `work_item_tags`: work_item, tag
|
||||
- `infrastructure_tags`: infrastructure, tag
|
||||
|
||||
---
|
||||
|
||||
## File Locations
|
||||
|
||||
All models created in: `D:\ClaudeTools\api\models\`
|
||||
|
||||
```
|
||||
api/models/
|
||||
├── backup_log.py ✅ NEW
|
||||
├── environmental_insight.py ✅ NEW
|
||||
├── external_integration.py ✅ NEW
|
||||
├── infrastructure_tag.py ✅ NEW
|
||||
├── integration_credential.py ✅ NEW
|
||||
├── ticket_link.py ✅ NEW
|
||||
├── work_item_tag.py ✅ NEW
|
||||
└── __init__.py ✅ UPDATED
|
||||
```
|
||||
|
||||
### Updated __init__.py
|
||||
|
||||
Added all 7 new models to imports and `__all__` list for proper package exposure.
|
||||
|
||||
---
|
||||
|
||||
## Missing Tables - Recommendation
|
||||
|
||||
**Action Required:** Clarify with project lead or spec author:
|
||||
|
||||
1. Should `environmental_examples` and `learning_metrics` be added to spec?
|
||||
2. Should `backup_schedules` be added for proactive backup planning?
|
||||
3. Should `api_users` and `api_tokens` be added, or is JWT-only auth sufficient?
|
||||
4. Is `infrastructure_tags` junction table correct (not explicitly in spec)?
|
||||
|
||||
If these tables are needed, they should be:
|
||||
- Added to MSP-MODE-SPEC.md with full schema definitions
|
||||
- Assigned to a coding agent for implementation
|
||||
|
||||
---
|
||||
|
||||
## Testing Recommendations
|
||||
|
||||
1. **Verify Foreign Keys**: Ensure `clients`, `infrastructure`, `sessions`, `work_items`, `tags`, and `failure_patterns` tables exist before creating these models.
|
||||
|
||||
2. **Encryption Testing**: Test `integration_credentials` encryption/decryption with actual AES-256-GCM implementation.
|
||||
|
||||
3. **Duration Calculation**: Test `backup_log.calculate_duration()` method with various time ranges.
|
||||
|
||||
4. **Junction Tables**: Verify CASCADE deletes work correctly for `work_item_tags` and `infrastructure_tags`.
|
||||
|
||||
5. **Index Performance**: Test query performance on indexed columns with realistic data volumes.
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. ✅ Models created and added to package
|
||||
2. ⏳ Clarify missing 5 tables with project lead
|
||||
3. ⏳ Create Alembic migrations for these 7 tables
|
||||
4. ⏳ Add relationship definitions after all models complete
|
||||
5. ⏳ Write unit tests for models
|
||||
6. ⏳ Test with actual MariaDB schema creation
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
**Completed:** 7 of 12 assigned models
|
||||
**Reason for Incomplete:** 5 tables not found in MSP-MODE-SPEC.md specification
|
||||
**Quality:** All created models are production-ready, follow SQLAlchemy 2.0 best practices, and match spec exactly
|
||||
**Blockers:** Need clarification on missing table definitions
|
||||
|
||||
**Agent #4 Status:** Ready for next assignment or specification updates
|
||||
34
docs/archives/agent-deliveries/AGENT4_SUMMARY.md
Normal file
34
docs/archives/agent-deliveries/AGENT4_SUMMARY.md
Normal file
@@ -0,0 +1,34 @@
|
||||
# Agent #4 - Quick Summary
|
||||
|
||||
## Assignment
|
||||
Create 12 models: Context Learning + Integrations + Backup + API + Junction
|
||||
|
||||
## Delivered
|
||||
**7 of 12 models** - All production-ready, spec-compliant
|
||||
|
||||
### ✅ Created Models
|
||||
1. `environmental_insight.py` - Environmental insights (context learning)
|
||||
2. `external_integration.py` - External system interactions log
|
||||
3. `integration_credential.py` - Encrypted OAuth/API credentials
|
||||
4. `ticket_link.py` - Session ↔ external tickets
|
||||
5. `backup_log.py` - Database backup tracking
|
||||
6. `work_item_tag.py` - Work items ↔ tags junction
|
||||
7. `infrastructure_tag.py` - Infrastructure ↔ tags junction
|
||||
|
||||
### ❌ Missing from Spec (Not Created)
|
||||
- `environmental_examples` - No definition found
|
||||
- `learning_metrics` - No definition found
|
||||
- `backup_schedules` - No definition found
|
||||
- `api_users` - No definition found
|
||||
- `api_tokens` - No definition found
|
||||
|
||||
## Status
|
||||
✅ All created models pass Python syntax validation
|
||||
✅ All models use SQLAlchemy 2.0 patterns
|
||||
✅ All indexes and constraints match spec
|
||||
✅ Package __init__.py updated with new models
|
||||
|
||||
## Action Required
|
||||
Clarify missing 5 tables - should they be added to spec?
|
||||
|
||||
See `AGENT4_DELIVERY.md` for full details.
|
||||
485
docs/archives/agent-deliveries/AUTOCODER_EXTRACTION_REPORT.md
Normal file
485
docs/archives/agent-deliveries/AUTOCODER_EXTRACTION_REPORT.md
Normal file
@@ -0,0 +1,485 @@
|
||||
# AutoCoder Resources Extraction Report
|
||||
|
||||
**Date:** 2026-01-17
|
||||
**Source:** AutoCoder project (Autocode-remix fork)
|
||||
**Destination:** D:\ClaudeTools
|
||||
**Status:** Successfully Completed
|
||||
|
||||
---
|
||||
|
||||
## Extraction Summary
|
||||
|
||||
Successfully extracted and organized MCP servers, commands, skills, and templates from the imported AutoCoder project into ClaudeTools.
|
||||
|
||||
**Total Items Extracted:** 13 files across 4 categories
|
||||
|
||||
---
|
||||
|
||||
## Files Extracted
|
||||
|
||||
### 1. Commands (3 files)
|
||||
|
||||
**Location:** `D:\ClaudeTools\.claude\commands\`
|
||||
|
||||
| File | Size | Source | Purpose |
|
||||
|------|------|--------|---------|
|
||||
| `checkpoint.md` | 1.8 KB | AutoCoder | Create development checkpoint with commit |
|
||||
| `create-spec.md` | 19 KB | AutoCoder | Create app specification for autonomous coding |
|
||||
| `sync.md` | 6.0 KB | Existing | Cross-machine context synchronization |
|
||||
|
||||
**New Commands:** 2 (checkpoint, create-spec)
|
||||
**Existing Commands:** 1 (sync)
|
||||
|
||||
---
|
||||
|
||||
### 2. Skills (2 files)
|
||||
|
||||
**Location:** `D:\ClaudeTools\.claude\skills\frontend-design\`
|
||||
|
||||
| File | Size | Source | Purpose |
|
||||
|------|------|--------|---------|
|
||||
| `SKILL.md` | 4.4 KB | AutoCoder | Frontend design skill definition |
|
||||
| `LICENSE.txt` | 10 KB | AutoCoder | Skill license information |
|
||||
|
||||
**New Skills:** 1 (frontend-design)
|
||||
|
||||
---
|
||||
|
||||
### 3. Templates (4 files)
|
||||
|
||||
**Location:** `D:\ClaudeTools\.claude\templates\`
|
||||
|
||||
| File | Size | Source | Purpose |
|
||||
|------|------|--------|---------|
|
||||
| `app_spec.template.txt` | 8.9 KB | AutoCoder | Application specification template |
|
||||
| `coding_prompt.template.md` | 14 KB | AutoCoder | Standard autonomous coding prompt |
|
||||
| `coding_prompt_yolo.template.md` | 7.8 KB | AutoCoder | Fast-paced coding prompt |
|
||||
| `initializer_prompt.template.md` | 19 KB | AutoCoder | Project initialization prompt |
|
||||
|
||||
**New Templates:** 4 (all new - directory created)
|
||||
|
||||
---
|
||||
|
||||
### 4. MCP Server (4 files)
|
||||
|
||||
**Location:** `D:\ClaudeTools\mcp-servers\feature-management\`
|
||||
|
||||
| File | Size | Source | Purpose |
|
||||
|------|------|--------|---------|
|
||||
| `feature_mcp.py` | 14 KB | AutoCoder | Feature management MCP server |
|
||||
| `__init__.py` | 49 bytes | AutoCoder | Python module marker |
|
||||
| `README.md` | 11 KB | Created | Server documentation |
|
||||
| `config.example.json` | 2.6 KB | Created | Configuration example |
|
||||
|
||||
**New MCP Servers:** 1 (feature-management)
|
||||
|
||||
---
|
||||
|
||||
## Directory Structure Created
|
||||
|
||||
```
|
||||
D:\ClaudeTools/
|
||||
├── .claude/
|
||||
│ ├── commands/
|
||||
│ │ ├── sync.md [EXISTING]
|
||||
│ │ ├── create-spec.md [NEW - AutoCoder]
|
||||
│ │ └── checkpoint.md [NEW - AutoCoder]
|
||||
│ │
|
||||
│ ├── skills/ [NEW DIRECTORY]
|
||||
│ │ └── frontend-design/ [NEW - AutoCoder]
|
||||
│ │ ├── SKILL.md
|
||||
│ │ └── LICENSE.txt
|
||||
│ │
|
||||
│ └── templates/ [NEW DIRECTORY]
|
||||
│ ├── app_spec.template.txt [NEW - AutoCoder]
|
||||
│ ├── coding_prompt.template.md [NEW - AutoCoder]
|
||||
│ ├── coding_prompt_yolo.template.md [NEW - AutoCoder]
|
||||
│ └── initializer_prompt.template.md [NEW - AutoCoder]
|
||||
│
|
||||
└── mcp-servers/ [NEW DIRECTORY]
|
||||
└── feature-management/ [NEW - AutoCoder]
|
||||
├── feature_mcp.py [AutoCoder]
|
||||
├── __init__.py [AutoCoder]
|
||||
├── README.md [Created]
|
||||
└── config.example.json [Created]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Documentation Created
|
||||
|
||||
### 1. AUTOCODER_INTEGRATION.md
|
||||
|
||||
**Location:** `D:\ClaudeTools\AUTOCODER_INTEGRATION.md`
|
||||
**Size:** Comprehensive integration guide
|
||||
|
||||
**Contents:**
|
||||
- Overview of extracted resources
|
||||
- Directory structure
|
||||
- Detailed documentation for each command
|
||||
- Detailed documentation for each skill
|
||||
- Detailed documentation for each template
|
||||
- MCP server setup guide
|
||||
- Typical autonomous coding workflow
|
||||
- Integration with ClaudeTools API
|
||||
- Configuration examples
|
||||
- Testing procedures
|
||||
- Troubleshooting guide
|
||||
- Best practices
|
||||
- Migration notes
|
||||
|
||||
---
|
||||
|
||||
### 2. MCP Server README
|
||||
|
||||
**Location:** `D:\ClaudeTools\mcp-servers\feature-management\README.md`
|
||||
**Size:** 11 KB
|
||||
|
||||
**Contents:**
|
||||
- MCP server overview
|
||||
- Architecture details
|
||||
- Database schema
|
||||
- All 8 available tools documented
|
||||
- Installation & configuration
|
||||
- Typical workflow examples
|
||||
- Integration with ClaudeTools
|
||||
- Troubleshooting
|
||||
- Differences from REST API
|
||||
|
||||
---
|
||||
|
||||
### 3. MCP Server Config Example
|
||||
|
||||
**Location:** `D:\ClaudeTools\mcp-servers\feature-management\config.example.json`
|
||||
**Size:** 2.6 KB
|
||||
|
||||
**Contents:**
|
||||
- Example Claude Desktop configuration
|
||||
- Platform-specific examples (Windows, macOS, Linux)
|
||||
- Virtual environment examples
|
||||
- Full configuration example with multiple MCP servers
|
||||
|
||||
---
|
||||
|
||||
### 4. Updated CLAUDE.md
|
||||
|
||||
**Location:** `D:\ClaudeTools\.claude\CLAUDE.md`
|
||||
**Changes:**
|
||||
- Updated project structure to show new directories
|
||||
- Added AutoCoder resources to Important Files section
|
||||
- Added available commands to Quick Reference
|
||||
- Added available skills to Quick Reference
|
||||
- Added reference to AUTOCODER_INTEGRATION.md
|
||||
|
||||
---
|
||||
|
||||
## Source Information
|
||||
|
||||
### Original Source Location
|
||||
```
|
||||
C:\Users\MikeSwanson\claude-projects\Autocode-remix\Autocode-fork\autocoder-master\
|
||||
├── .claude/
|
||||
│ ├── commands/
|
||||
│ │ ├── checkpoint.md
|
||||
│ │ ├── create-spec.md
|
||||
│ │ └── import-spec.md [NOT COPIED - not requested]
|
||||
│ ├── skills/
|
||||
│ │ └── frontend-design/
|
||||
│ └── templates/
|
||||
└── mcp_server/
|
||||
├── feature_mcp.py
|
||||
└── __init__.py
|
||||
```
|
||||
|
||||
### Conversation History
|
||||
- **Location:** `D:\ClaudeTools\imported-conversations\auto-claude-variants\autocode-remix-fork\`
|
||||
- **Files:** 85 JSONL conversation files
|
||||
- **Size:** 37 MB
|
||||
|
||||
---
|
||||
|
||||
## Verification
|
||||
|
||||
### File Integrity Check
|
||||
|
||||
All files verified successfully:
|
||||
|
||||
```
|
||||
Commands: 3 files ✓
|
||||
Skills: 2 files ✓
|
||||
Templates: 4 files ✓
|
||||
MCP Server: 4 files ✓
|
||||
Documentation: 4 files ✓
|
||||
-----------------------------------
|
||||
Total: 17 files ✓
|
||||
```
|
||||
|
||||
### File Permissions
|
||||
|
||||
- All `.md` files: readable (644)
|
||||
- All `.txt` files: readable (644)
|
||||
- All `.py` files: executable (755)
|
||||
- All `.json` files: readable (644)
|
||||
|
||||
---
|
||||
|
||||
## How to Activate Each Component
|
||||
|
||||
### Commands
|
||||
|
||||
**Already active** - No configuration needed
|
||||
|
||||
Commands are automatically available in Claude Code:
|
||||
```bash
|
||||
/create-spec # Create app specification
|
||||
/checkpoint # Create development checkpoint
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Skills
|
||||
|
||||
**Already active** - No configuration needed
|
||||
|
||||
Skills are automatically available in Claude Code:
|
||||
```bash
|
||||
/frontend-design # Activate frontend design skill
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Templates
|
||||
|
||||
**Already active** - Used internally by commands
|
||||
|
||||
Templates are used by:
|
||||
- `/create-spec` uses `app_spec.template.txt`
|
||||
- Autonomous coding agents use `coding_prompt.template.md`
|
||||
- Fast prototyping uses `coding_prompt_yolo.template.md`
|
||||
- Project initialization uses `initializer_prompt.template.md`
|
||||
|
||||
---
|
||||
|
||||
### MCP Server
|
||||
|
||||
**Requires configuration**
|
||||
|
||||
#### Step 1: Install Dependencies
|
||||
|
||||
```bash
|
||||
# Activate virtual environment
|
||||
D:\ClaudeTools\venv\Scripts\activate
|
||||
|
||||
# Install required packages
|
||||
pip install fastmcp sqlalchemy pydantic
|
||||
```
|
||||
|
||||
#### Step 2: Configure Claude Desktop
|
||||
|
||||
Edit Claude Desktop configuration file:
|
||||
- **Windows:** `%APPDATA%\Claude\claude_desktop_config.json`
|
||||
|
||||
Add this configuration:
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"features": {
|
||||
"command": "python",
|
||||
"args": ["D:\\ClaudeTools\\mcp-servers\\feature-management\\feature_mcp.py"],
|
||||
"env": {
|
||||
"PROJECT_DIR": "D:\\ClaudeTools\\projects\\your-project"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### Step 3: Restart Claude Desktop
|
||||
|
||||
Close and reopen Claude Desktop for changes to take effect.
|
||||
|
||||
#### Step 4: Verify
|
||||
|
||||
You should now have access to these MCP tools:
|
||||
- `feature_get_stats`
|
||||
- `feature_get_next`
|
||||
- `feature_mark_passing`
|
||||
- `feature_mark_in_progress`
|
||||
- `feature_skip`
|
||||
- `feature_clear_in_progress`
|
||||
- `feature_get_for_regression`
|
||||
- `feature_create_bulk`
|
||||
|
||||
**Full setup guide:** See `AUTOCODER_INTEGRATION.md`
|
||||
|
||||
---
|
||||
|
||||
## Integration Points with ClaudeTools
|
||||
|
||||
### 1. Context Recall System
|
||||
|
||||
Feature completions can be logged to the context recall system:
|
||||
|
||||
```python
|
||||
POST /api/conversation-contexts
|
||||
{
|
||||
"context_type": "feature_completion",
|
||||
"title": "Completed Feature: User Authentication",
|
||||
"dense_summary": "Implemented JWT-based authentication...",
|
||||
"tags": ["authentication", "feature", "jwt"]
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Decision Logging
|
||||
|
||||
Architectural decisions can be tracked:
|
||||
|
||||
```python
|
||||
POST /api/decision-logs
|
||||
{
|
||||
"decision_type": "technical",
|
||||
"decision_text": "Use JWT for authentication",
|
||||
"rationale": "Stateless, scalable, industry standard",
|
||||
"tags": ["authentication", "architecture"]
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Session Tracking
|
||||
|
||||
Feature work can be tracked with sessions:
|
||||
|
||||
```python
|
||||
POST /api/sessions
|
||||
{
|
||||
"project_id": "uuid",
|
||||
"metadata": {"feature_id": 15, "feature_name": "User login"}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Testing the Integration
|
||||
|
||||
### Test Commands
|
||||
|
||||
```bash
|
||||
# Test create-spec
|
||||
/create-spec
|
||||
# Should display specification creation interface
|
||||
|
||||
# Test checkpoint
|
||||
/checkpoint
|
||||
# Should create git commit and save context
|
||||
```
|
||||
|
||||
### Test Skills
|
||||
|
||||
```bash
|
||||
# Test frontend-design
|
||||
/frontend-design
|
||||
# Should activate frontend design mode
|
||||
```
|
||||
|
||||
### Test MCP Server (after configuration)
|
||||
|
||||
```python
|
||||
# In Claude Code with MCP server running
|
||||
|
||||
# Test stats
|
||||
feature_get_stats()
|
||||
# Should return progress statistics
|
||||
|
||||
# Test get next
|
||||
feature_get_next()
|
||||
# Should return next feature or empty queue message
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Benefits
|
||||
|
||||
### For ClaudeTools
|
||||
|
||||
1. **Autonomous Coding Support:** Full workflow for spec-driven development
|
||||
2. **Feature Tracking:** Priority-based feature queue management
|
||||
3. **Quality Control:** Checkpoint system with context preservation
|
||||
4. **Design Patterns:** Frontend design skill for modern UI development
|
||||
5. **Templates:** Structured prompts for consistent agent behavior
|
||||
|
||||
### For Development Workflow
|
||||
|
||||
1. **Spec-Driven:** Start with clear requirements (`/create-spec`)
|
||||
2. **Trackable:** Monitor progress with feature management
|
||||
3. **Recoverable:** Checkpoints preserve context at key moments
|
||||
4. **Consistent:** Templates ensure agents follow best practices
|
||||
5. **Specialized:** Skills provide domain expertise (frontend design)
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
### Recommended Actions
|
||||
|
||||
1. **Try the commands:**
|
||||
- Run `/create-spec` on a test project
|
||||
- Create a checkpoint with `/checkpoint`
|
||||
|
||||
2. **Set up MCP server:**
|
||||
- Install dependencies
|
||||
- Configure Claude Desktop
|
||||
- Test feature management tools
|
||||
|
||||
3. **Integrate with existing workflows:**
|
||||
- Use `/checkpoint` after completing features
|
||||
- Log feature completions to context recall
|
||||
- Track decisions with decision_logs API
|
||||
|
||||
4. **Customize templates:**
|
||||
- Review templates in `.claude/templates/`
|
||||
- Adjust to match your coding style
|
||||
- Add project-specific requirements
|
||||
|
||||
---
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- **Integration Guide:** `AUTOCODER_INTEGRATION.md` (comprehensive guide)
|
||||
- **MCP Server Docs:** `mcp-servers/feature-management/README.md`
|
||||
- **MCP Config Example:** `mcp-servers/feature-management/config.example.json`
|
||||
- **ClaudeTools Docs:** `.claude/CLAUDE.md` (updated)
|
||||
- **Context Recall:** `.claude/CONTEXT_RECALL_QUICK_START.md`
|
||||
|
||||
---
|
||||
|
||||
## Version History
|
||||
|
||||
| Date | Version | Action |
|
||||
|------|---------|--------|
|
||||
| 2026-01-17 | 1.0 | Initial extraction from AutoCoder |
|
||||
|
||||
---
|
||||
|
||||
## Completion Checklist
|
||||
|
||||
- [x] Created new directory structure
|
||||
- [x] Copied 2 commands from AutoCoder
|
||||
- [x] Copied 1 skill from AutoCoder
|
||||
- [x] Copied 4 templates from AutoCoder
|
||||
- [x] Copied MCP server files from AutoCoder
|
||||
- [x] Created comprehensive README for MCP server
|
||||
- [x] Created configuration example for MCP server
|
||||
- [x] Created AUTOCODER_INTEGRATION.md guide
|
||||
- [x] Updated main CLAUDE.md documentation
|
||||
- [x] Verified all files copied correctly
|
||||
- [x] Documented activation procedures
|
||||
- [x] Created extraction report (this file)
|
||||
|
||||
---
|
||||
|
||||
**Extraction Status:** Complete
|
||||
**Total Duration:** ~15 minutes
|
||||
**Files Processed:** 13 source files + 4 documentation files
|
||||
**Success Rate:** 100%
|
||||
|
||||
**Last Updated:** 2026-01-17
|
||||
@@ -0,0 +1,35 @@
|
||||
# Context Export Results
|
||||
|
||||
**Date:** 2026-01-18
|
||||
**Status:** No contexts to export
|
||||
|
||||
## Summary
|
||||
|
||||
Attempted to export tombstoned and database contexts before removing the context system.
|
||||
|
||||
## Findings
|
||||
|
||||
1. **Tombstone Files:** 0 found in `imported-conversations/` directory
|
||||
2. **API Status:** Not running (http://172.16.3.30:8001 returned 404)
|
||||
3. **Contexts Exported:** 0
|
||||
|
||||
## Conclusion
|
||||
|
||||
No tombstoned or database contexts exist to preserve. The context system can be safely removed without data loss.
|
||||
|
||||
## Export Script
|
||||
|
||||
Created `scripts/export-tombstoned-contexts.py` for future use if needed before removal is finalized.
|
||||
|
||||
To run export manually (requires API running):
|
||||
```bash
|
||||
# Export all database contexts
|
||||
python scripts/export-tombstoned-contexts.py --export-all
|
||||
|
||||
# Export only tombstoned contexts
|
||||
python scripts/export-tombstoned-contexts.py
|
||||
```
|
||||
|
||||
## Next Steps
|
||||
|
||||
Proceeding with context system removal as planned.
|
||||
@@ -0,0 +1,150 @@
|
||||
# Context System Removal - COMPLETE
|
||||
|
||||
**Date:** 2026-01-18
|
||||
**Status:** ✅ COMPLETE (Code removed, database preserved)
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
Successfully removed the entire conversation context/recall system code from ClaudeTools while preserving the database tables for safety.
|
||||
|
||||
---
|
||||
|
||||
## What Was Removed
|
||||
|
||||
### ✅ All Code Components (80+ files)
|
||||
|
||||
**API Layer:**
|
||||
- 4 routers (35+ endpoints)
|
||||
- 4 services
|
||||
- 4 schemas
|
||||
- 5 models
|
||||
|
||||
**Infrastructure:**
|
||||
- 13 Claude Code hooks (user-prompt-submit, task-complete, etc.)
|
||||
- 15+ scripts (import, migration, testing)
|
||||
- 5 test files
|
||||
|
||||
**Documentation:**
|
||||
- 30+ markdown files
|
||||
- All context-related guides and references
|
||||
|
||||
**Files Modified:**
|
||||
- api/main.py
|
||||
- api/models/__init__.py
|
||||
- api/schemas/__init__.py
|
||||
- api/services/__init__.py
|
||||
- .claude/claude.md (completely rewritten)
|
||||
|
||||
---
|
||||
|
||||
## ⚠️ Database Tables PRESERVED
|
||||
|
||||
The following tables remain in the database for safety:
|
||||
- `conversation_contexts`
|
||||
- `context_snippets`
|
||||
- `context_tags`
|
||||
- `project_states`
|
||||
- `decision_logs`
|
||||
|
||||
**Why Preserved:**
|
||||
- Safety net in case any data is needed
|
||||
- No code exists to access them (orphaned tables)
|
||||
- Can be dropped later when confirmed not needed
|
||||
|
||||
**To Drop Later (Optional):**
|
||||
```bash
|
||||
cd D:/ClaudeTools
|
||||
alembic upgrade head # Applies migration 20260118_172743
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Impact
|
||||
|
||||
**Files Deleted:** 80+
|
||||
**Files Modified:** 5
|
||||
**Code Lines Removed:** 5,000+
|
||||
**API Endpoints Removed:** 35+
|
||||
**Database Tables:** 5 (preserved for safety)
|
||||
|
||||
---
|
||||
|
||||
## System State
|
||||
|
||||
**Before Removal:**
|
||||
- 130 endpoints across 21 entities
|
||||
- 43 database tables
|
||||
- Context recall system active
|
||||
|
||||
**After Removal:**
|
||||
- 95 endpoints across 17 entities
|
||||
- 38 active tables + 5 orphaned context tables
|
||||
- Context recall system completely removed from code
|
||||
|
||||
---
|
||||
|
||||
## Migration Available
|
||||
|
||||
A migration has been created to drop the tables when ready:
|
||||
- **File:** `migrations/versions/20260118_172743_remove_context_system.py`
|
||||
- **Action:** Drops all 5 context tables
|
||||
- **Status:** NOT APPLIED (preserved for safety)
|
||||
|
||||
---
|
||||
|
||||
## Documentation Created
|
||||
|
||||
1. **CONTEXT_SYSTEM_REMOVAL_SUMMARY.md** - Detailed removal report
|
||||
2. **CONTEXT_EXPORT_RESULTS.md** - Export attempt results
|
||||
3. **CONTEXT_SYSTEM_REMOVAL_COMPLETE.md** - This file (final status)
|
||||
4. **scripts/export-tombstoned-contexts.py** - Export script (if needed later)
|
||||
|
||||
---
|
||||
|
||||
## Verification
|
||||
|
||||
**Code Verified:**
|
||||
- ✅ No import errors in api/main.py
|
||||
- ✅ All context imports removed from __init__.py files
|
||||
- ✅ Hooks directory cleaned
|
||||
- ✅ Scripts directory cleaned
|
||||
- ✅ Documentation updated
|
||||
|
||||
**Database:**
|
||||
- ✅ Tables still exist (preserved)
|
||||
- ✅ No code can access them (orphaned)
|
||||
- ⏳ Can be dropped when confirmed not needed
|
||||
|
||||
---
|
||||
|
||||
## Next Steps (If Needed)
|
||||
|
||||
**To Drop Database Tables Later:**
|
||||
```bash
|
||||
# When absolutely sure data is not needed:
|
||||
cd D:/ClaudeTools
|
||||
alembic upgrade head
|
||||
```
|
||||
|
||||
**To Restore System (Emergency):**
|
||||
1. Restore deleted files from git history
|
||||
2. Re-add router registrations to api/main.py
|
||||
3. Re-add imports to __init__.py files
|
||||
4. Database tables already exist (no migration needed)
|
||||
|
||||
---
|
||||
|
||||
## Notes
|
||||
|
||||
- **No tombstoned contexts found** - system was not actively used
|
||||
- **No data loss** - all database tables preserved
|
||||
- **Clean codebase** - all references removed
|
||||
- **Safe rollback** - git history preserves everything
|
||||
|
||||
---
|
||||
|
||||
**Removal Completed:** 2026-01-18
|
||||
**Database Preserved:** Yes (5 tables orphaned but safe)
|
||||
**Ready for Production:** Yes (all code references removed)
|
||||
@@ -0,0 +1,235 @@
|
||||
# Context System Removal Summary
|
||||
|
||||
**Date:** 2026-01-18
|
||||
**Status:** Complete (pending database migration)
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Successfully removed the entire conversation context/recall system from ClaudeTools, including all database tables, API endpoints, models, services, hooks, scripts, and documentation.
|
||||
|
||||
---
|
||||
|
||||
## What Was Removed
|
||||
|
||||
### Database Tables (5 tables)
|
||||
- `conversation_contexts` - Main context storage
|
||||
- `context_snippets` - Knowledge fragments
|
||||
- `context_tags` - Normalized tags table
|
||||
- `project_states` - Project state tracking
|
||||
- `decision_logs` - Decision documentation
|
||||
|
||||
### API Layer (35+ endpoints)
|
||||
**Routers Deleted:**
|
||||
- `api/routers/conversation_contexts.py`
|
||||
- `api/routers/context_snippets.py`
|
||||
- `api/routers/project_states.py`
|
||||
- `api/routers/decision_logs.py`
|
||||
|
||||
**Services Deleted:**
|
||||
- `api/services/conversation_context_service.py`
|
||||
- `api/services/context_snippet_service.py`
|
||||
- `api/services/project_state_service.py`
|
||||
- `api/services/decision_log_service.py`
|
||||
|
||||
**Schemas Deleted:**
|
||||
- `api/schemas/conversation_context.py`
|
||||
- `api/schemas/context_snippet.py`
|
||||
- `api/schemas/project_state.py`
|
||||
- `api/schemas/decision_log.py`
|
||||
|
||||
### Models (5 models)
|
||||
- `api/models/conversation_context.py`
|
||||
- `api/models/context_snippet.py`
|
||||
- `api/models/context_tag.py`
|
||||
- `api/models/decision_log.py`
|
||||
- `api/models/project_state.py`
|
||||
|
||||
### Claude Code Hooks (13 files)
|
||||
- `user-prompt-submit` (and variants)
|
||||
- `task-complete` (and variants)
|
||||
- `sync-contexts`
|
||||
- `periodic-context-save` (and variants)
|
||||
- Cache and queue directories
|
||||
|
||||
### Scripts (15+ files)
|
||||
- `import-conversations.py`
|
||||
- `check-tombstones.py`
|
||||
- `migrate_tags_to_normalized_table.py`
|
||||
- `verify_tag_migration.py`
|
||||
- And 11+ more...
|
||||
|
||||
### Utilities
|
||||
- `api/utils/context_compression.py`
|
||||
- `api/utils/CONTEXT_COMPRESSION_*.md` (3 files)
|
||||
|
||||
### Test Files (5 files)
|
||||
- `test_context_recall_system.py`
|
||||
- `test_context_compression_quick.py`
|
||||
- `test_recall_search_fix.py`
|
||||
- `test_recall_search_simple.py`
|
||||
- `test_recall_diagnostic.py`
|
||||
|
||||
### Documentation (30+ files)
|
||||
**Root Directory:**
|
||||
- All `CONTEXT_RECALL_*.md` files (10 files)
|
||||
- All `CONTEXT_TAGS_*.md` files (4 files)
|
||||
- All `CONTEXT_SAVE_*.md` files (3 files)
|
||||
- `RECALL_SEARCH_FIX_SUMMARY.md`
|
||||
- `CONVERSATION_IMPORT_SUMMARY.md`
|
||||
- `TOMBSTONE_*.md` files (2 files)
|
||||
|
||||
**.claude Directory:**
|
||||
- `CONTEXT_RECALL_*.md` (2 files)
|
||||
- `PERIODIC_CONTEXT_SAVE.md`
|
||||
- `SCHEMA_CONTEXT.md`
|
||||
- `SNAPSHOT_*.md` (2 files)
|
||||
- `commands/snapshot*` (3 files)
|
||||
|
||||
**scripts Directory:**
|
||||
- `CONVERSATION_IMPORT_README.md`
|
||||
- `IMPORT_QUICK_START.md`
|
||||
- `IMPORT_COMMANDS.txt`
|
||||
- `TOMBSTONE_QUICK_START.md`
|
||||
|
||||
**migrations Directory:**
|
||||
- `README_CONTEXT_TAGS.md`
|
||||
- `apply_performance_indexes.sql`
|
||||
|
||||
### Migrations
|
||||
**Deleted (original creation migrations):**
|
||||
- `a0dfb0b4373c_add_context_recall_models.py`
|
||||
- `20260118_132847_add_context_tags_normalized_table.py`
|
||||
|
||||
**Created (removal migration):**
|
||||
- `20260118_172743_remove_context_system.py`
|
||||
|
||||
---
|
||||
|
||||
## Files Modified
|
||||
|
||||
### 1. api/main.py
|
||||
- Removed context router imports (4 lines)
|
||||
- Removed router registrations (4 lines)
|
||||
|
||||
### 2. api/models/__init__.py
|
||||
- Removed 5 model imports
|
||||
- Removed 5 model exports from __all__
|
||||
|
||||
### 3. api/schemas/__init__.py
|
||||
- Removed 4 schema imports
|
||||
- Removed 16 schema exports from __all__
|
||||
|
||||
### 4. api/services/__init__.py
|
||||
- Removed 4 service imports
|
||||
- Removed 4 service exports from __all__
|
||||
|
||||
### 5. .claude/claude.md
|
||||
- **Completely rewritten** - removed all context system references
|
||||
- Removed Context Recall System section
|
||||
- Removed context-related endpoints
|
||||
- Removed context-related workflows
|
||||
- Removed context documentation references
|
||||
- Removed token optimization section
|
||||
- Removed context troubleshooting
|
||||
- Updated Quick Facts and Recent Work sections
|
||||
|
||||
---
|
||||
|
||||
## Export Results
|
||||
|
||||
**Tombstone Files Found:** 0
|
||||
**Database Contexts Exported:** 0 (API not running)
|
||||
**Conclusion:** No tombstoned or database contexts existed to preserve
|
||||
|
||||
**Export Script Created:** `scripts/export-tombstoned-contexts.py` (for future use if needed)
|
||||
|
||||
---
|
||||
|
||||
## Remaining Work
|
||||
|
||||
### Database Migration
|
||||
The database migration has been created but NOT yet applied:
|
||||
```bash
|
||||
# To apply the migration and drop the tables:
|
||||
cd D:/ClaudeTools
|
||||
alembic upgrade head
|
||||
```
|
||||
|
||||
**WARNING:** This will permanently delete all context data from the database.
|
||||
|
||||
### Known Remaining References
|
||||
The following files still contain references to context services but are not critical:
|
||||
- `api/routers/bulk_import.py` - May have context imports (needs cleanup)
|
||||
- `api/routers/version.py` - References deleted files in version info
|
||||
- `api/utils/__init__.py` - May have context utility exports
|
||||
|
||||
These can be cleaned up as needed.
|
||||
|
||||
---
|
||||
|
||||
## Impact Summary
|
||||
|
||||
**Total Files Deleted:** 80+ files
|
||||
**Files Modified:** 5 files
|
||||
**Database Tables to Drop:** 5 tables
|
||||
**API Endpoints Removed:** 35+ endpoints
|
||||
**Lines of Code Removed:** 5,000+ lines
|
||||
|
||||
---
|
||||
|
||||
## Verification Steps
|
||||
|
||||
### 1. Code Verification
|
||||
```bash
|
||||
# Search for remaining references
|
||||
grep -r "conversation_context\|context_snippet\|decision_log\|project_state\|context_tag" api/ --include="*.py"
|
||||
```
|
||||
|
||||
### 2. Database Verification (after migration)
|
||||
```bash
|
||||
# Connect to database
|
||||
mysql -h 172.16.3.30 -u claudetools -p claudetools
|
||||
|
||||
# Verify tables are dropped
|
||||
SHOW TABLES LIKE '%context%';
|
||||
SHOW TABLES LIKE '%decision%';
|
||||
SHOW TABLES LIKE '%snippet%';
|
||||
# Should return no results
|
||||
```
|
||||
|
||||
### 3. API Verification
|
||||
```bash
|
||||
# Start API
|
||||
python -m api.main
|
||||
|
||||
# Check OpenAPI docs
|
||||
# Visit http://localhost:8000/api/docs
|
||||
# Verify no context-related endpoints appear
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
If issues arise:
|
||||
1. **Code restoration:** Restore deleted files from git history
|
||||
2. **Database restoration:** Restore from database backup OR re-run original migrations
|
||||
3. **Hook restoration:** Restore hook files from git history
|
||||
4. **Router restoration:** Re-add router registrations in main.py
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. **Apply database migration** to drop tables (when ready)
|
||||
2. **Clean up remaining references** in bulk_import.py, version.py, and utils/__init__.py
|
||||
3. **Test API startup** to ensure no import errors
|
||||
4. **Update SESSION_STATE.md** to reflect the removal
|
||||
5. **Create git commit** documenting the removal
|
||||
|
||||
---
|
||||
|
||||
**Last Updated:** 2026-01-18
|
||||
**Removal Status:** Code cleanup complete, database migration pending
|
||||
98
docs/archives/deployment-old/MANUAL_DEPLOY_SIMPLE.txt
Normal file
98
docs/archives/deployment-old/MANUAL_DEPLOY_SIMPLE.txt
Normal file
@@ -0,0 +1,98 @@
|
||||
================================================================================
|
||||
MANUAL DEPLOYMENT - Interactive SSH Session
|
||||
================================================================================
|
||||
|
||||
Step 1: Open SSH Connection
|
||||
----------------------------
|
||||
In PowerShell, run:
|
||||
|
||||
plink guru@172.16.3.30
|
||||
|
||||
Enter your password. You should see:
|
||||
guru@gururmm:~$
|
||||
|
||||
|
||||
Step 2: Check if file was copied
|
||||
---------------------------------
|
||||
In the SSH session, type:
|
||||
|
||||
ls -lh /tmp/conv.py
|
||||
|
||||
If it says "No such file or directory":
|
||||
- Exit SSH (type: exit)
|
||||
- Run: pscp D:\ClaudeTools\api\routers\conversation_contexts.py guru@172.16.3.30:/tmp/conv.py
|
||||
- Reconnect: plink guru@172.16.3.30
|
||||
- Continue below
|
||||
|
||||
If file exists, continue:
|
||||
|
||||
|
||||
Step 3: Deploy the file
|
||||
------------------------
|
||||
In the SSH session, run these commands one at a time:
|
||||
|
||||
sudo mv /tmp/conv.py /opt/claudetools/api/routers/conversation_contexts.py
|
||||
sudo chown claudetools:claudetools /opt/claudetools/api/routers/conversation_contexts.py
|
||||
sudo systemctl restart claudetools-api
|
||||
|
||||
(sudo should not ask for password if passwordless is set up)
|
||||
|
||||
|
||||
Step 4: Verify deployment
|
||||
--------------------------
|
||||
In the SSH session, run:
|
||||
|
||||
grep -c "search_term.*Query" /opt/claudetools/api/routers/conversation_contexts.py
|
||||
|
||||
Expected output: 1 (or higher)
|
||||
If you see 0, the old file is still there.
|
||||
|
||||
|
||||
Step 5: Check service status
|
||||
-----------------------------
|
||||
In the SSH session, run:
|
||||
|
||||
sudo systemctl status claudetools-api --no-pager | head -15
|
||||
|
||||
Look for:
|
||||
- "Active: active (running)"
|
||||
- Recent timestamp (today's date, last few minutes)
|
||||
|
||||
|
||||
Step 6: Exit SSH
|
||||
-----------------
|
||||
Type:
|
||||
|
||||
exit
|
||||
|
||||
|
||||
Step 7: Test the API
|
||||
---------------------
|
||||
Back in PowerShell, run:
|
||||
|
||||
python -c "import requests; r=requests.get('http://172.16.3.30:8001/api/conversation-contexts/recall', headers={'Authorization': 'Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJpbXBvcnQtc2NyaXB0Iiwic2NvcGVzIjpbImFkbWluIiwiaW1wb3J0Il0sImV4cCI6MTc3MTI3NTEyOX0.-DJF50tq0MaNwVQBdO7cGYNuO5pQuXte-tTj5DpHi2U'}, params={'search_term': 'dataforth', 'limit': 2}); data=r.json(); print('SUCCESS - New code!' if 'contexts' in data else 'FAILED - Old code'); print(f'Contexts: {len(data.get(\"contexts\", []))}' if 'contexts' in data else f'Format: {list(data.keys())}')"
|
||||
|
||||
Expected output if successful:
|
||||
SUCCESS - New code!
|
||||
Contexts: 2
|
||||
|
||||
Expected output if failed:
|
||||
FAILED - Old code
|
||||
Format: ['context', 'project_id', 'tags', 'limit', 'min_relevance_score']
|
||||
|
||||
|
||||
================================================================================
|
||||
ALTERNATIVE: Copy/Paste File Content
|
||||
================================================================================
|
||||
|
||||
If pscp isn't working, you can manually paste the file content:
|
||||
|
||||
1. Open: D:\ClaudeTools\api\routers\conversation_contexts.py in a text editor
|
||||
2. Copy ALL the content (Ctrl+A, Ctrl+C)
|
||||
3. SSH to server: plink guru@172.16.3.30
|
||||
4. Create file with nano: nano /tmp/conv.py
|
||||
5. Paste content (Right-click in PuTTY)
|
||||
6. Save: Ctrl+X, Y, Enter
|
||||
7. Continue from Step 3 above
|
||||
|
||||
================================================================================
|
||||
26
docs/archives/deployment-old/QUICK_DEPLOY.txt
Normal file
26
docs/archives/deployment-old/QUICK_DEPLOY.txt
Normal file
@@ -0,0 +1,26 @@
|
||||
================================================================================
|
||||
QUICK DEPLOYMENT - Run These 2 Commands
|
||||
================================================================================
|
||||
|
||||
STEP 1: Copy the file (in PowerShell)
|
||||
--------------------------------------
|
||||
pscp D:\ClaudeTools\api\routers\conversation_contexts.py guru@172.16.3.30:/tmp/conv.py
|
||||
|
||||
(Enter password once)
|
||||
|
||||
|
||||
STEP 2: Deploy and restart (in PowerShell)
|
||||
-------------------------------------------
|
||||
plink guru@172.16.3.30 "sudo mv /tmp/conv.py /opt/claudetools/api/routers/conversation_contexts.py && sudo chown claudetools:claudetools /opt/claudetools/api/routers/conversation_contexts.py && sudo systemctl restart claudetools-api && sleep 3 && echo 'Deployed!' && grep -c 'search_term.*Query' /opt/claudetools/api/routers/conversation_contexts.py"
|
||||
|
||||
(Enter password once - sudo should be passwordless after that)
|
||||
Expected output: "Deployed!" followed by "1"
|
||||
|
||||
|
||||
STEP 3: Test (in PowerShell)
|
||||
-----------------------------
|
||||
python -c "import requests; r=requests.get('http://172.16.3.30:8001/api/conversation-contexts/recall', headers={'Authorization': 'Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJpbXBvcnQtc2NyaXB0Iiwic2NvcGVzIjpbImFkbWluIiwiaW1wb3J0Il0sImV4cCI6MTc3MTI3NTEyOX0.-DJF50tq0MaNwVQBdO7cGYNuO5pQuXte-tTj5DpHi2U'}, params={'search_term': 'dataforth', 'limit': 2}); print('SUCCESS!' if 'contexts' in r.json() else 'Failed'); print(f\"Found {len(r.json().get('contexts', []))} contexts\" if 'contexts' in r.json() else '')"
|
||||
|
||||
Expected: "SUCCESS!" and "Found 2 contexts"
|
||||
|
||||
================================================================================
|
||||
@@ -0,0 +1,219 @@
|
||||
# Periodic Save Task - Invisible Mode Setup
|
||||
|
||||
## Problem Solved
|
||||
The `periodic_save_check.py` Task Scheduler task was showing a flashing console window every minute. This has been fixed by configuring the task to run completely invisibly.
|
||||
|
||||
---
|
||||
|
||||
## What Changed
|
||||
|
||||
### 1. Updated Setup Script
|
||||
**File:** `D:\ClaudeTools\.claude\hooks\setup_periodic_save.ps1`
|
||||
|
||||
**Changes:**
|
||||
- Uses `pythonw.exe` instead of `python.exe` (no console window)
|
||||
- Added `-Hidden` flag to task settings
|
||||
- Changed LogonType from `Interactive` to `S4U` (Service-For-User = background)
|
||||
- Added verification instructions in output
|
||||
|
||||
### 2. Created Update Script
|
||||
**File:** `D:\ClaudeTools\.claude\hooks\update_to_invisible.ps1`
|
||||
|
||||
**Purpose:**
|
||||
- Quick one-command update for existing tasks
|
||||
- Preserves existing triggers and settings
|
||||
- Validates pythonw.exe exists
|
||||
- Shows verification output
|
||||
|
||||
### 3. Created Documentation
|
||||
**File:** `D:\ClaudeTools\.claude\PERIODIC_SAVE_INVISIBLE_SETUP.md`
|
||||
|
||||
**Contents:**
|
||||
- Automatic setup instructions
|
||||
- Manual update procedures (PowerShell and GUI)
|
||||
- Verification steps
|
||||
- Troubleshooting guide
|
||||
|
||||
---
|
||||
|
||||
## How to Fix Your Current Task
|
||||
|
||||
### Option 1: Automatic (Recommended)
|
||||
|
||||
Run the update script:
|
||||
|
||||
```powershell
|
||||
# From PowerShell in D:\ClaudeTools
|
||||
.\.claude\hooks\update_to_invisible.ps1
|
||||
```
|
||||
|
||||
This will:
|
||||
- Find pythonw.exe automatically
|
||||
- Update the task to use pythonw.exe
|
||||
- Set the task to run hidden
|
||||
- Verify all settings are correct
|
||||
|
||||
### Option 2: Recreate Task
|
||||
|
||||
Re-run the setup script (removes old task and creates new one):
|
||||
|
||||
```powershell
|
||||
# From PowerShell in D:\ClaudeTools
|
||||
.\.claude\hooks\setup_periodic_save.ps1
|
||||
```
|
||||
|
||||
### Option 3: Manual (GUI)
|
||||
|
||||
1. Open Task Scheduler (Win + R → `taskschd.msc`)
|
||||
2. Find "ClaudeTools - Periodic Context Save"
|
||||
3. Right-click → Properties
|
||||
4. **Actions tab:** Change `python.exe` to `pythonw.exe`
|
||||
5. **General tab:** Check "Hidden" checkbox
|
||||
6. Click OK
|
||||
|
||||
---
|
||||
|
||||
## Verification
|
||||
|
||||
After updating, verify the task is configured correctly:
|
||||
|
||||
```powershell
|
||||
# Quick verification
|
||||
Get-ScheduledTask -TaskName "ClaudeTools - Periodic Context Save" |
|
||||
Select-Object -ExpandProperty Actions |
|
||||
Select-Object Execute
|
||||
|
||||
# Should show: ...pythonw.exe (NOT python.exe)
|
||||
|
||||
# Check hidden setting
|
||||
Get-ScheduledTask -TaskName "ClaudeTools - Periodic Context Save" |
|
||||
Select-Object -ExpandProperty Settings |
|
||||
Select-Object Hidden
|
||||
|
||||
# Should show: Hidden: True
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Technical Details
|
||||
|
||||
### pythonw.exe vs python.exe
|
||||
|
||||
| Executable | Console Window | Use Case |
|
||||
|------------|---------------|----------|
|
||||
| `python.exe` | Shows console | Interactive scripts, debugging |
|
||||
| `pythonw.exe` | No console | Background tasks, GUI apps |
|
||||
|
||||
### Task Scheduler Settings
|
||||
|
||||
| Setting | Old Value | New Value | Purpose |
|
||||
|---------|-----------|-----------|---------|
|
||||
| Executable | python.exe | pythonw.exe | No console window |
|
||||
| Hidden | False | True | Hide from task list |
|
||||
| LogonType | Interactive | S4U | Run in background |
|
||||
|
||||
### What is S4U (Service-For-User)?
|
||||
|
||||
- Runs tasks in background session
|
||||
- No interactive window
|
||||
- Doesn't require user to be logged in
|
||||
- Ideal for background automation
|
||||
|
||||
---
|
||||
|
||||
## Files Modified/Created
|
||||
|
||||
### Modified
|
||||
- `D:\ClaudeTools\.claude\hooks\setup_periodic_save.ps1`
|
||||
- Lines 9-18: Auto-detect pythonw.exe path
|
||||
- Line 29: Use pythonw.exe instead of python.exe
|
||||
- Line 43: Added `-Hidden` flag
|
||||
- Line 46: Changed LogonType to S4U
|
||||
- Lines 59-64: Updated output messages
|
||||
|
||||
### Created
|
||||
- `D:\ClaudeTools\.claude\hooks\update_to_invisible.ps1`
|
||||
- Quick update script for existing tasks
|
||||
|
||||
- `D:\ClaudeTools\.claude\PERIODIC_SAVE_INVISIBLE_SETUP.md`
|
||||
- Complete setup and troubleshooting guide
|
||||
|
||||
- `D:\ClaudeTools\INVISIBLE_PERIODIC_SAVE_SUMMARY.md`
|
||||
- This file - quick reference summary
|
||||
|
||||
---
|
||||
|
||||
## Testing
|
||||
|
||||
After updating, the task will run every minute but you should see:
|
||||
- ✓ No console window flashing
|
||||
- ✓ No visible task execution
|
||||
- ✓ Logs still being written to `D:\ClaudeTools\.claude\periodic-save.log`
|
||||
|
||||
Check logs to verify it's working:
|
||||
|
||||
```powershell
|
||||
Get-Content D:\ClaudeTools\.claude\periodic-save.log -Tail 20
|
||||
```
|
||||
|
||||
You should see log entries appearing every minute (when Claude is active) without any visible window.
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Still seeing console window?
|
||||
|
||||
**Check executable:**
|
||||
```powershell
|
||||
Get-ScheduledTask -TaskName "ClaudeTools - Periodic Context Save" |
|
||||
Select-Object -ExpandProperty Actions
|
||||
```
|
||||
- If shows `python.exe` - update didn't work, try manual update
|
||||
- If shows `pythonw.exe` - should be invisible (check hidden setting)
|
||||
|
||||
**Check hidden setting:**
|
||||
```powershell
|
||||
Get-ScheduledTask -TaskName "ClaudeTools - Periodic Context Save" |
|
||||
Select-Object -ExpandProperty Settings |
|
||||
Select-Object Hidden
|
||||
```
|
||||
- Should show `Hidden: True`
|
||||
- If False, run update script again
|
||||
|
||||
**Check LogonType:**
|
||||
```powershell
|
||||
Get-ScheduledTask -TaskName "ClaudeTools - Periodic Context Save" |
|
||||
Select-Object -ExpandProperty Principal
|
||||
```
|
||||
- Should show `LogonType: S4U`
|
||||
- If Interactive, run update script again
|
||||
|
||||
### pythonw.exe not found?
|
||||
|
||||
```powershell
|
||||
# Check Python installation
|
||||
Get-Command python | Select-Object -ExpandProperty Source
|
||||
|
||||
# Check if pythonw.exe exists in same directory
|
||||
$PythonPath = (Get-Command python).Source
|
||||
$PythonDir = Split-Path $PythonPath -Parent
|
||||
Test-Path (Join-Path $PythonDir "pythonw.exe")
|
||||
```
|
||||
|
||||
If False, reinstall Python. pythonw.exe should always come with Python on Windows.
|
||||
|
||||
---
|
||||
|
||||
## Current Status
|
||||
|
||||
**Task Name:** ClaudeTools - Periodic Context Save
|
||||
**Frequency:** Every 1 minute
|
||||
**Action:** Check activity, save context every 5 minutes of active work
|
||||
**Visibility:** Hidden (no console window)
|
||||
**Logs:** `D:\ClaudeTools\.claude\periodic-save.log`
|
||||
|
||||
---
|
||||
|
||||
**Last Updated:** 2026-01-17
|
||||
**Updated Files:** 1 modified, 3 created
|
||||
728
docs/archives/offline-mode-removed/OFFLINE_MODE_COMPLETE.md
Normal file
728
docs/archives/offline-mode-removed/OFFLINE_MODE_COMPLETE.md
Normal file
@@ -0,0 +1,728 @@
|
||||
# Offline Mode Implementation - Complete ✅
|
||||
|
||||
**Date:** 2026-01-17
|
||||
**Status:** COMPLETE
|
||||
**Version:** 2.0 (Offline-Capable Context Recall)
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
ClaudeTools Context Recall System has been successfully upgraded to support **full offline operation** with automatic synchronization. The system now gracefully handles network outages, server maintenance, and connectivity issues without data loss.
|
||||
|
||||
---
|
||||
|
||||
## What Was Accomplished
|
||||
|
||||
### ✅ Complete Offline Support
|
||||
|
||||
**Before (V1):**
|
||||
- Context recall only worked when API was available
|
||||
- Contexts were silently lost when API failed
|
||||
- No fallback mechanism
|
||||
- No data resilience
|
||||
|
||||
**After (V2):**
|
||||
- **Offline Reading:** Falls back to local cache when API unavailable
|
||||
- **Offline Writing:** Queues contexts locally when API unavailable
|
||||
- **Automatic Sync:** Background synchronization when API restored
|
||||
- **Zero Data Loss:** All contexts preserved and eventually uploaded
|
||||
|
||||
### ✅ Infrastructure Created
|
||||
|
||||
**New Directories:**
|
||||
```
|
||||
.claude/
|
||||
├── context-cache/ # Downloaded contexts for offline reading
|
||||
│ └── [project-id]/
|
||||
│ ├── latest.json # Most recent contexts from API
|
||||
│ └── last_updated # Cache timestamp
|
||||
└── context-queue/ # Pending contexts to upload
|
||||
├── pending/ # Contexts waiting to upload
|
||||
├── uploaded/ # Successfully synced (auto-cleaned)
|
||||
└── failed/ # Failed uploads (manual review needed)
|
||||
```
|
||||
|
||||
**Git Protection:**
|
||||
```gitignore
|
||||
# Added to .gitignore
|
||||
.claude/context-cache/
|
||||
.claude/context-queue/
|
||||
```
|
||||
|
||||
### ✅ Enhanced Hooks (V2)
|
||||
|
||||
**1. user-prompt-submit (v2)**
|
||||
- Tries API with 3-second timeout
|
||||
- Falls back to local cache if API unavailable
|
||||
- Shows clear "Offline Mode" warning
|
||||
- Updates cache on successful API fetch
|
||||
- **Location:** `.claude/hooks/user-prompt-submit`
|
||||
|
||||
**2. task-complete (v2)**
|
||||
- Tries API save with 5-second timeout
|
||||
- Queues locally if API unavailable
|
||||
- Triggers background sync (opportunistic)
|
||||
- Shows clear warning when queuing
|
||||
- **Location:** `.claude/hooks/task-complete`
|
||||
|
||||
**3. sync-contexts (new)**
|
||||
- Uploads queued contexts to API
|
||||
- Moves successful uploads to `uploaded/`
|
||||
- Moves failed uploads to `failed/`
|
||||
- Auto-cleans old uploaded contexts
|
||||
- Can run manually or automatically
|
||||
- **Location:** `.claude/hooks/sync-contexts`
|
||||
|
||||
### ✅ Documentation Created
|
||||
|
||||
1. **`.claude/OFFLINE_MODE.md`** (481 lines)
|
||||
- Complete architecture documentation
|
||||
- How it works (online, offline, sync modes)
|
||||
- Directory structure explanation
|
||||
- Usage guide with examples
|
||||
- Migration from V1 to V2
|
||||
- Troubleshooting guide
|
||||
- Performance & security considerations
|
||||
- FAQ section
|
||||
|
||||
2. **`OFFLINE_MODE_TEST_PROCEDURE.md`** (517 lines)
|
||||
- 5-phase test plan
|
||||
- Step-by-step instructions
|
||||
- Expected outputs documented
|
||||
- Results template
|
||||
- Quick reference commands
|
||||
- Troubleshooting section
|
||||
|
||||
3. **`OFFLINE_MODE_VERIFICATION.md`** (520+ lines)
|
||||
- Component verification checklist
|
||||
- Before/after comparison
|
||||
- User experience examples
|
||||
- Security & privacy analysis
|
||||
- Readiness confirmation
|
||||
|
||||
4. **`scripts/upgrade-to-offline-mode.sh`** (170 lines)
|
||||
- Automated upgrade from V1 to V2
|
||||
- Backs up existing hooks
|
||||
- Creates directory structure
|
||||
- Updates .gitignore
|
||||
- Verifies installation
|
||||
|
||||
---
|
||||
|
||||
## How It Works
|
||||
|
||||
### Online Mode (Normal Operation)
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ User sends message to Claude Code │
|
||||
└────────────────┬────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ user-prompt-submit hook executes │
|
||||
├─────────────────────────────────────────────────────────┤
|
||||
│ 1. Fetch context from API (http://172.16.3.30:8001) │
|
||||
│ 2. Save response to cache (.claude/context-cache/) │
|
||||
│ 3. Update timestamp (last_updated) │
|
||||
│ 4. Inject context into conversation │
|
||||
└────────────────┬────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ Claude processes request with context │
|
||||
└────────────────┬────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ Task completes │
|
||||
└────────────────┬────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ task-complete hook executes │
|
||||
├─────────────────────────────────────────────────────────┤
|
||||
│ 1. POST context to API │
|
||||
│ 2. Receive success (HTTP 200/201) │
|
||||
│ 3. Display: "✓ Context saved to database" │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Offline Mode (API Unavailable)
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ User sends message to Claude Code │
|
||||
└────────────────┬────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ user-prompt-submit hook executes │
|
||||
├─────────────────────────────────────────────────────────┤
|
||||
│ 1. Try API fetch → TIMEOUT after 3 seconds │
|
||||
│ 2. Fall back to local cache │
|
||||
│ 3. Read: .claude/context-cache/[project]/latest.json │
|
||||
│ 4. Inject cached context with warning │
|
||||
│ "⚠️ Offline Mode - Using cached context" │
|
||||
└────────────────┬────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ Claude processes with cached context │
|
||||
└────────────────┬────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ Task completes │
|
||||
└────────────────┬────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ task-complete hook executes │
|
||||
├─────────────────────────────────────────────────────────┤
|
||||
│ 1. Try POST to API → TIMEOUT after 5 seconds │
|
||||
│ 2. Queue locally to pending/ │
|
||||
│ 3. Save: pending/[project]_[timestamp]_context.json │
|
||||
│ 4. Display: "⚠ Context queued locally" │
|
||||
│ 5. Trigger background sync (opportunistic) │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Sync Mode (API Restored)
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ API becomes available again │
|
||||
└────────────────┬────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ Next user interaction OR manual sync command │
|
||||
└────────────────┬────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ sync-contexts script executes (background) │
|
||||
├─────────────────────────────────────────────────────────┤
|
||||
│ 1. Scan .claude/context-queue/pending/*.json │
|
||||
│ 2. For each queued context: │
|
||||
│ - POST to API with JWT auth │
|
||||
│ - On success: move to uploaded/ │
|
||||
│ - On failure: move to failed/ │
|
||||
│ 3. Clean up uploaded/ (keep last 100) │
|
||||
│ 4. Display sync summary │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## User Experience
|
||||
|
||||
### Scenario 1: Working Online
|
||||
|
||||
```
|
||||
You: "Add a new feature to the API"
|
||||
|
||||
[Hook fetches context from API in < 1 second]
|
||||
[Context injected - Claude remembers previous work]
|
||||
|
||||
Claude: "I'll add that feature. I see from our previous session
|
||||
that we're using FastAPI with SQLAlchemy 2.0..."
|
||||
|
||||
[Task completes]
|
||||
[Hook saves context to API]
|
||||
Message: "✓ Context saved to database"
|
||||
```
|
||||
|
||||
### Scenario 2: Working Offline
|
||||
|
||||
```
|
||||
You: "Continue working on the API"
|
||||
|
||||
[API unavailable - hook uses cache]
|
||||
Message: "⚠️ Offline Mode - Using cached context (API unavailable)"
|
||||
|
||||
Claude: "I'll continue the work. Based on cached context from
|
||||
2 hours ago, we were implementing the authentication
|
||||
endpoints..."
|
||||
|
||||
[Task completes]
|
||||
[Hook queues context locally]
|
||||
Message: "⚠ Context queued locally (API unavailable) - will sync when online"
|
||||
|
||||
[Later, when API restored]
|
||||
[Background sync automatically uploads queued context]
|
||||
Message: "✓ Synced 1 context(s)"
|
||||
```
|
||||
|
||||
### Scenario 3: First Run (No Cache)
|
||||
|
||||
```
|
||||
You: "Help me with this project"
|
||||
|
||||
[No cache exists yet, hook exits silently]
|
||||
|
||||
Claude: "I'd be happy to help! Tell me more about your project..."
|
||||
|
||||
[Task completes]
|
||||
[Hook saves context to API - cache created]
|
||||
Message: "✓ Context saved to database"
|
||||
|
||||
[Next time, context will be available]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Key Features
|
||||
|
||||
### 1. Intelligent Fallback
|
||||
|
||||
- **3-second API timeout** for context fetch (user-prompt-submit)
|
||||
- **5-second API timeout** for context save (task-complete)
|
||||
- **Immediate fallback** to local cache/queue
|
||||
- **No blocking** - user never waits for failed API calls
|
||||
|
||||
### 2. Zero Data Loss
|
||||
|
||||
- **Cache persists** until replaced by newer API fetch
|
||||
- **Queue persists** until successfully uploaded
|
||||
- **Failed uploads** moved to `failed/` for manual review
|
||||
- **Automatic retry** on next sync attempt
|
||||
|
||||
### 3. Transparent Operation
|
||||
|
||||
- **Clear warnings** when using cache ("Offline Mode")
|
||||
- **Clear warnings** when queuing ("will sync when online")
|
||||
- **Success messages** when online ("Context saved to database")
|
||||
- **Sync summaries** showing upload results
|
||||
|
||||
### 4. Automatic Maintenance
|
||||
|
||||
- **Background sync** triggered on next user interaction
|
||||
- **Auto-cleanup** of uploaded contexts (keeps last 100)
|
||||
- **Cache refresh** on every successful API call
|
||||
- **No manual intervention** required
|
||||
|
||||
---
|
||||
|
||||
## Testing Status
|
||||
|
||||
### ✅ Component Verification Complete
|
||||
|
||||
All components have been installed and verified:
|
||||
|
||||
1. ✅ **V2 Hooks Installed**
|
||||
- user-prompt-submit (v2 with offline support)
|
||||
- task-complete (v2 with offline support)
|
||||
- sync-contexts (new sync script)
|
||||
|
||||
2. ✅ **Directory Structure Created**
|
||||
- .claude/context-cache/ (for offline reading)
|
||||
- .claude/context-queue/pending/ (for queued saves)
|
||||
- .claude/context-queue/uploaded/ (successful syncs)
|
||||
- .claude/context-queue/failed/ (failed syncs)
|
||||
|
||||
3. ✅ **Configuration Updated**
|
||||
- API URL: http://172.16.3.30:8001 (centralized)
|
||||
- .gitignore: cache and queue excluded
|
||||
|
||||
4. ✅ **API Health Verified**
|
||||
- API online and healthy
|
||||
- Database connected
|
||||
- Endpoints accessible
|
||||
|
||||
### 📋 Live Testing Procedure Available
|
||||
|
||||
Complete test procedure documented in `OFFLINE_MODE_TEST_PROCEDURE.md`:
|
||||
|
||||
**Test Phases:**
|
||||
1. Phase 1: Baseline (online mode verification)
|
||||
2. Phase 2: Offline mode (cache fallback test)
|
||||
3. Phase 3: Context queuing (save fallback test)
|
||||
4. Phase 4: Automatic sync (restore and upload test)
|
||||
5. Phase 5: Cache refresh (force refresh test)
|
||||
|
||||
**To run tests:**
|
||||
```bash
|
||||
# Review test procedure
|
||||
cat OFFLINE_MODE_TEST_PROCEDURE.md
|
||||
|
||||
# When ready, follow phase-by-phase instructions
|
||||
# (Requires SSH access to stop/start API)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Usage
|
||||
|
||||
### Normal Operation (No Action Required)
|
||||
|
||||
The system works automatically - no commands needed:
|
||||
|
||||
1. **Open Claude Code** in any ClaudeTools directory
|
||||
2. **Send messages** - context recalled automatically
|
||||
3. **Complete tasks** - context saved automatically
|
||||
4. **Work offline** - system falls back gracefully
|
||||
5. **Go back online** - system syncs automatically
|
||||
|
||||
### Manual Commands (Optional)
|
||||
|
||||
**Force sync queued contexts:**
|
||||
```bash
|
||||
bash .claude/hooks/sync-contexts
|
||||
```
|
||||
|
||||
**View cached context:**
|
||||
```bash
|
||||
PROJECT_ID=$(git config --local claude.projectid)
|
||||
cat .claude/context-cache/$PROJECT_ID/latest.json | python -m json.tool
|
||||
```
|
||||
|
||||
**Check queue status:**
|
||||
```bash
|
||||
ls -la .claude/context-queue/pending/ # Waiting to upload
|
||||
ls -la .claude/context-queue/uploaded/ # Successfully synced
|
||||
ls -la .claude/context-queue/failed/ # Need review
|
||||
```
|
||||
|
||||
**Clear cache (force refresh):**
|
||||
```bash
|
||||
PROJECT_ID=$(git config --local claude.projectid)
|
||||
rm -rf .claude/context-cache/$PROJECT_ID
|
||||
# Next message will fetch fresh context from API
|
||||
```
|
||||
|
||||
**Manual sync with output:**
|
||||
```bash
|
||||
bash .claude/hooks/sync-contexts
|
||||
|
||||
# Example output:
|
||||
# ===================================
|
||||
# Syncing Queued Contexts
|
||||
# ===================================
|
||||
# Found 2 pending context(s)
|
||||
#
|
||||
# Processing: claudetools_20260117_140122_context.json
|
||||
# ✓ Uploaded successfully
|
||||
# Processing: claudetools_20260117_141533_state.json
|
||||
# ✓ Uploaded successfully
|
||||
#
|
||||
# ===================================
|
||||
# Sync Complete
|
||||
# ===================================
|
||||
# Successful: 2
|
||||
# Failed: 0
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Architecture Benefits
|
||||
|
||||
### 1. Data Resilience
|
||||
|
||||
**Problem Solved:**
|
||||
- Network outages no longer cause data loss
|
||||
- Server maintenance doesn't interrupt work
|
||||
- Connectivity issues handled gracefully
|
||||
|
||||
**How:**
|
||||
- Local cache preserves last known state
|
||||
- Local queue preserves unsaved changes
|
||||
- Automatic sync when restored
|
||||
|
||||
### 2. Improved User Experience
|
||||
|
||||
**Problem Solved:**
|
||||
- Silent failures confused users
|
||||
- No feedback when offline
|
||||
- Lost work when API down
|
||||
|
||||
**How:**
|
||||
- Clear "Offline Mode" warnings
|
||||
- Status messages for all operations
|
||||
- Transparent fallback behavior
|
||||
|
||||
### 3. Centralized Architecture Compatible
|
||||
|
||||
**Problem Solved:**
|
||||
- Centralized API requires network
|
||||
- Single point of failure
|
||||
- No local redundancy
|
||||
|
||||
**How:**
|
||||
- Local cache provides redundancy
|
||||
- Queue enables async operation
|
||||
- Works with or without network
|
||||
|
||||
### 4. Zero Configuration
|
||||
|
||||
**Problem Solved:**
|
||||
- Complex setup procedures
|
||||
- Manual intervention needed
|
||||
- User doesn't understand system
|
||||
|
||||
**How:**
|
||||
- Automatic detection of offline state
|
||||
- Automatic fallback and sync
|
||||
- Transparent operation
|
||||
|
||||
---
|
||||
|
||||
## Security & Privacy
|
||||
|
||||
### What's Cached Locally
|
||||
|
||||
**Safe to Cache:**
|
||||
- ✅ Context summaries (compressed, not full transcripts)
|
||||
- ✅ Titles and tags
|
||||
- ✅ Relevance scores
|
||||
- ✅ Project IDs (hashes)
|
||||
- ✅ Timestamps
|
||||
|
||||
**Never Cached:**
|
||||
- ❌ JWT tokens (in separate config file)
|
||||
- ❌ Database credentials
|
||||
- ❌ User passwords
|
||||
- ❌ Full conversation transcripts
|
||||
- ❌ Sensitive credential data
|
||||
|
||||
### Git Protection
|
||||
|
||||
```gitignore
|
||||
# Automatically added to .gitignore
|
||||
.claude/context-cache/ # Local cache - don't commit
|
||||
.claude/context-queue/ # Local queue - don't commit
|
||||
```
|
||||
|
||||
**Result:** No accidental commits of local data
|
||||
|
||||
### File Permissions
|
||||
|
||||
- Directories created with user-only access
|
||||
- No group or world readable permissions
|
||||
- Only current user can access cache/queue
|
||||
|
||||
### Cleanup
|
||||
|
||||
- **Uploaded queue:** Auto-cleaned (keeps last 100)
|
||||
- **Cache:** Replaced on each API fetch
|
||||
- **Failed:** Manual review available
|
||||
|
||||
---
|
||||
|
||||
## What Changed in Your System
|
||||
|
||||
### Before This Session
|
||||
|
||||
**System:**
|
||||
- V1 hooks (API-only, no fallback)
|
||||
- No local storage
|
||||
- Silent failures
|
||||
- Data loss when offline
|
||||
|
||||
**User Experience:**
|
||||
- "Where did my context go?"
|
||||
- "Why doesn't Claude remember?"
|
||||
- "The API was down, I lost everything"
|
||||
|
||||
### After This Session
|
||||
|
||||
**System:**
|
||||
- V2 hooks (offline-capable)
|
||||
- Local cache and queue
|
||||
- Clear warnings and status
|
||||
- Zero data loss
|
||||
|
||||
**User Experience:**
|
||||
- "Working offline - using cached context"
|
||||
- "Context queued - will sync later"
|
||||
- "Everything synced automatically"
|
||||
|
||||
---
|
||||
|
||||
## Files Created/Modified
|
||||
|
||||
### Created (New Files)
|
||||
|
||||
1. `.claude/hooks/sync-contexts` - Sync script
|
||||
2. `.claude/OFFLINE_MODE.md` - Architecture docs
|
||||
3. `OFFLINE_MODE_TEST_PROCEDURE.md` - Test guide
|
||||
4. `OFFLINE_MODE_VERIFICATION.md` - Verification report
|
||||
5. `OFFLINE_MODE_COMPLETE.md` - This summary
|
||||
6. `scripts/upgrade-to-offline-mode.sh` - Upgrade script
|
||||
7. `.claude/context-cache/` - Cache directory (empty)
|
||||
8. `.claude/context-queue/` - Queue directories (empty)
|
||||
|
||||
### Modified (Updated Files)
|
||||
|
||||
1. `.claude/hooks/user-prompt-submit` - Upgraded to v2
|
||||
2. `.claude/hooks/task-complete` - Upgraded to v2
|
||||
3. `.gitignore` - Added cache and queue exclusions
|
||||
|
||||
### Backed Up (Previous Versions)
|
||||
|
||||
The upgrade script creates backups automatically:
|
||||
- `.claude/hooks/backup_[timestamp]/user-prompt-submit` (v1)
|
||||
- `.claude/hooks/backup_[timestamp]/task-complete` (v1)
|
||||
|
||||
---
|
||||
|
||||
## Performance Impact
|
||||
|
||||
### Storage
|
||||
|
||||
- **Cache per project:** ~10-50 KB
|
||||
- **Queue per context:** ~1-2 KB
|
||||
- **Total impact:** Negligible (< 1 MB typical)
|
||||
|
||||
### Speed
|
||||
|
||||
- **Cache read:** < 100ms (instant)
|
||||
- **Queue write:** < 100ms (instant)
|
||||
- **Sync per context:** ~0.5 seconds
|
||||
- **Background sync:** Non-blocking
|
||||
|
||||
### Network
|
||||
|
||||
- **API timeout (read):** 3 seconds max
|
||||
- **API timeout (write):** 5 seconds max
|
||||
- **Sync traffic:** Minimal (POST requests only)
|
||||
|
||||
**Result:** No noticeable performance impact
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
### System is Ready for Production Use
|
||||
|
||||
**No action required** - the system is fully operational:
|
||||
|
||||
1. ✅ All components installed
|
||||
2. ✅ All hooks upgraded to v2
|
||||
3. ✅ All documentation complete
|
||||
4. ✅ API verified healthy
|
||||
5. ✅ Configuration correct
|
||||
|
||||
### Optional: Live Testing
|
||||
|
||||
If you want to verify offline mode works:
|
||||
|
||||
1. Review test procedure:
|
||||
```bash
|
||||
cat OFFLINE_MODE_TEST_PROCEDURE.md
|
||||
```
|
||||
|
||||
2. Run Phase 1 (Baseline):
|
||||
- Use Claude Code normally
|
||||
- Verify cache created
|
||||
|
||||
3. Run Phase 2-4 (Offline Test):
|
||||
- Stop API: `ssh guru@172.16.3.30 sudo systemctl stop claudetools-api`
|
||||
- Use Claude Code (verify cache fallback)
|
||||
- Restart API: `ssh guru@172.16.3.30 sudo systemctl start claudetools-api`
|
||||
- Verify sync
|
||||
|
||||
### Optional: Setup Other Machines
|
||||
|
||||
When setting up ClaudeTools on another machine:
|
||||
|
||||
```bash
|
||||
# Clone repo
|
||||
git clone [repo-url] D:\ClaudeTools
|
||||
cd D:\ClaudeTools
|
||||
|
||||
# Run 30-second setup
|
||||
bash scripts/setup-new-machine.sh
|
||||
|
||||
# Done! Offline support included automatically
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Support & Troubleshooting
|
||||
|
||||
### Quick Diagnostics
|
||||
|
||||
**Check system status:**
|
||||
```bash
|
||||
# Verify v2 hooks installed
|
||||
head -3 .claude/hooks/user-prompt-submit # Should show "v2 - with offline support"
|
||||
|
||||
# Check API health
|
||||
curl -s http://172.16.3.30:8001/health # Should show {"status":"healthy"}
|
||||
|
||||
# Check cache exists
|
||||
ls -la .claude/context-cache/
|
||||
|
||||
# Check queue
|
||||
ls -la .claude/context-queue/pending/
|
||||
```
|
||||
|
||||
### Common Issues
|
||||
|
||||
**Issue:** Offline mode not activating
|
||||
```bash
|
||||
# Verify v2 hooks installed
|
||||
grep "v2 - with offline support" .claude/hooks/user-prompt-submit
|
||||
# If not found, run: bash scripts/upgrade-to-offline-mode.sh
|
||||
```
|
||||
|
||||
**Issue:** Contexts not syncing
|
||||
```bash
|
||||
# Check JWT token exists
|
||||
grep JWT_TOKEN .claude/context-recall-config.env
|
||||
|
||||
# Run manual sync
|
||||
bash .claude/hooks/sync-contexts
|
||||
```
|
||||
|
||||
**Issue:** Cache is stale
|
||||
```bash
|
||||
# Clear cache to force refresh
|
||||
PROJECT_ID=$(git config --local claude.projectid)
|
||||
rm -rf .claude/context-cache/$PROJECT_ID
|
||||
# Next Claude Code message will fetch fresh
|
||||
```
|
||||
|
||||
### Documentation References
|
||||
|
||||
- **Architecture:** `.claude/OFFLINE_MODE.md`
|
||||
- **Testing:** `OFFLINE_MODE_TEST_PROCEDURE.md`
|
||||
- **Verification:** `OFFLINE_MODE_VERIFICATION.md`
|
||||
- **Setup:** `scripts/upgrade-to-offline-mode.sh`
|
||||
|
||||
---
|
||||
|
||||
## Conclusion
|
||||
|
||||
### ✅ Mission Accomplished
|
||||
|
||||
Your request has been fully completed:
|
||||
|
||||
> "Verify all the local code to make sure it complies with the new setup for dynamic storage and retrieval of context and all other data. Also verify it has a fallback to local storage with a complete sync once database is functional."
|
||||
|
||||
**Completed:**
|
||||
1. ✅ Verified local code complies with centralized API setup
|
||||
2. ✅ Implemented complete fallback to local storage (cache + queue)
|
||||
3. ✅ Implemented complete sync mechanism (automatic + manual)
|
||||
4. ✅ Verified all components installed and ready
|
||||
5. ✅ Created comprehensive documentation
|
||||
|
||||
### 🎯 Results
|
||||
|
||||
**ClaudeTools Context Recall System v2.0:**
|
||||
- **Status:** Production Ready
|
||||
- **Offline Support:** Fully Implemented
|
||||
- **Data Loss:** Zero
|
||||
- **User Action Required:** None
|
||||
- **Documentation:** Complete
|
||||
|
||||
The system now provides **enterprise-grade reliability** with automatic offline fallback and seamless synchronization. Context is never lost, even during network outages or server maintenance.
|
||||
|
||||
---
|
||||
|
||||
**Implementation Date:** 2026-01-17
|
||||
**System Version:** 2.0 (Offline-Capable)
|
||||
**Status:** ✅ COMPLETE AND OPERATIONAL
|
||||
@@ -0,0 +1,445 @@
|
||||
# Offline Mode Test Procedure
|
||||
|
||||
**Version:** 2.0
|
||||
**Date:** 2026-01-17
|
||||
**System Status:** ✅ All Components Installed and Ready
|
||||
|
||||
---
|
||||
|
||||
## Pre-Test Verification (COMPLETED)
|
||||
|
||||
### ✅ Infrastructure Check
|
||||
```bash
|
||||
# Verified directories exist
|
||||
ls -la .claude/context-cache/ # ✅ Exists
|
||||
ls -la .claude/context-queue/ # ✅ Exists (pending, uploaded, failed)
|
||||
|
||||
# Verified v2 hooks installed
|
||||
head -3 .claude/hooks/user-prompt-submit # ✅ v2 with offline support
|
||||
head -3 .claude/hooks/task-complete # ✅ v2 with offline support
|
||||
head -3 .claude/hooks/sync-contexts # ✅ Sync script ready
|
||||
|
||||
# Verified configuration
|
||||
grep CLAUDE_API_URL .claude/context-recall-config.env
|
||||
# ✅ Output: CLAUDE_API_URL=http://172.16.3.30:8001
|
||||
|
||||
# Verified gitignore
|
||||
grep context-cache .gitignore # ✅ Present
|
||||
grep context-queue .gitignore # ✅ Present
|
||||
```
|
||||
|
||||
### ✅ Current System Status
|
||||
- **API:** http://172.16.3.30:8001 (ONLINE)
|
||||
- **Database:** 172.16.3.30:3306 (ONLINE)
|
||||
- **Health Check:** {"status":"healthy","database":"connected"}
|
||||
- **Hooks:** V2 (offline-capable)
|
||||
- **Storage:** Ready
|
||||
|
||||
---
|
||||
|
||||
## Test Procedure
|
||||
|
||||
### Phase 1: Baseline Test (Online Mode)
|
||||
|
||||
**Purpose:** Verify normal operation before testing offline
|
||||
|
||||
```bash
|
||||
# 1. Open Claude Code in D:\ClaudeTools
|
||||
cd D:\ClaudeTools
|
||||
|
||||
# 2. Send a test message to Claude
|
||||
# Expected output should include:
|
||||
# <!-- Context Recall: Retrieved X relevant context(s) from API -->
|
||||
# ## 📚 Previous Context
|
||||
|
||||
# 3. Check that context was cached
|
||||
PROJECT_ID=$(git config --local claude.projectid 2>/dev/null || git config --get remote.origin.url | md5sum | cut -d' ' -f1)
|
||||
ls -la .claude/context-cache/$PROJECT_ID/
|
||||
# Expected: latest.json and last_updated files
|
||||
|
||||
# 4. Verify cache contents
|
||||
cat .claude/context-cache/$PROJECT_ID/latest.json | python -m json.tool
|
||||
# Expected: Array of context objects with titles, summaries, scores
|
||||
```
|
||||
|
||||
**Success Criteria:**
|
||||
- ✅ Context retrieved from API
|
||||
- ✅ Cache file created with timestamp
|
||||
- ✅ Context injected into conversation
|
||||
|
||||
---
|
||||
|
||||
### Phase 2: Offline Mode Test (Cache Fallback)
|
||||
|
||||
**Purpose:** Verify system uses cached context when API unavailable
|
||||
|
||||
```bash
|
||||
# 1. SSH to RMM server
|
||||
ssh guru@172.16.3.30
|
||||
|
||||
# 2. Stop the API service
|
||||
sudo systemctl stop claudetools-api
|
||||
|
||||
# 3. Verify API is stopped
|
||||
sudo systemctl status claudetools-api --no-pager
|
||||
# Expected: Active: inactive (dead)
|
||||
|
||||
# 4. Exit SSH
|
||||
exit
|
||||
|
||||
# 5. Back on Windows - test context recall
|
||||
# Open Claude Code and send a message
|
||||
|
||||
# Expected output:
|
||||
# <!-- Context Recall: Retrieved X relevant context(s) from LOCAL CACHE (offline mode) -->
|
||||
# ## 📚 Previous Context
|
||||
# ⚠️ **Offline Mode** - Using cached context (API unavailable)
|
||||
```
|
||||
|
||||
**Success Criteria:**
|
||||
- ✅ Hook detects API unavailable
|
||||
- ✅ Falls back to cached context
|
||||
- ✅ Clear "Offline Mode" warning displayed
|
||||
- ✅ Conversation continues with cached context
|
||||
|
||||
---
|
||||
|
||||
### Phase 3: Context Queuing Test (Save Fallback)
|
||||
|
||||
**Purpose:** Verify contexts queue locally when API unavailable
|
||||
|
||||
```bash
|
||||
# 1. API should still be stopped from Phase 2
|
||||
|
||||
# 2. Complete a task in Claude Code
|
||||
# (This triggers task-complete hook)
|
||||
|
||||
# Expected stderr output:
|
||||
# ⚠ Context queued locally (API unavailable) - will sync when online
|
||||
|
||||
# 3. Check queue directory
|
||||
ls -la .claude/context-queue/pending/
|
||||
# Expected: One or more .json files with timestamp names
|
||||
# Example: claudetools_20260117_143022_context.json
|
||||
|
||||
# 4. View queued context
|
||||
cat .claude/context-queue/pending/*.json | python -m json.tool
|
||||
# Expected: JSON with project_id, context_type, title, dense_summary, etc.
|
||||
```
|
||||
|
||||
**Success Criteria:**
|
||||
- ✅ Context save attempt fails gracefully
|
||||
- ✅ Context queued in pending/ directory
|
||||
- ✅ User warned about offline queuing
|
||||
- ✅ No data loss
|
||||
|
||||
---
|
||||
|
||||
### Phase 4: Automatic Sync Test
|
||||
|
||||
**Purpose:** Verify queued contexts sync when API restored
|
||||
|
||||
```bash
|
||||
# 1. SSH to RMM server
|
||||
ssh guru@172.16.3.30
|
||||
|
||||
# 2. Start the API service
|
||||
sudo systemctl start claudetools-api
|
||||
|
||||
# 3. Verify API is running
|
||||
sudo systemctl status claudetools-api --no-pager
|
||||
# Expected: Active: active (running)
|
||||
|
||||
# 4. Test API health
|
||||
curl http://localhost:8001/health
|
||||
# Expected: {"status":"healthy","database":"connected"}
|
||||
|
||||
# 5. Exit SSH
|
||||
exit
|
||||
|
||||
# 6. Back on Windows - trigger sync
|
||||
# Method A: Send any message in Claude Code (automatic background sync)
|
||||
# Method B: Manual sync command
|
||||
bash .claude/hooks/sync-contexts
|
||||
|
||||
# Expected output from manual sync:
|
||||
# ===================================
|
||||
# Syncing Queued Contexts
|
||||
# ===================================
|
||||
# Found X pending context(s)
|
||||
#
|
||||
# Processing: [filename].json
|
||||
# ✓ Uploaded successfully
|
||||
#
|
||||
# ===================================
|
||||
# Sync Complete
|
||||
# ===================================
|
||||
# Successful: X
|
||||
# Failed: 0
|
||||
|
||||
# 7. Verify queue cleared
|
||||
ls -la .claude/context-queue/pending/
|
||||
# Expected: Empty (or nearly empty)
|
||||
|
||||
ls -la .claude/context-queue/uploaded/
|
||||
# Expected: Previously pending files moved here
|
||||
|
||||
# 8. Verify contexts in database
|
||||
curl -s "http://172.16.3.30:8001/api/conversation-contexts?limit=5" \
|
||||
-H "Authorization: Bearer $JWT_TOKEN" | python -m json.tool
|
||||
# Expected: Recently synced contexts appear in results
|
||||
```
|
||||
|
||||
**Success Criteria:**
|
||||
- ✅ Background sync triggered automatically
|
||||
- ✅ Queued contexts uploaded successfully
|
||||
- ✅ Files moved from pending/ to uploaded/
|
||||
- ✅ Contexts visible in database
|
||||
|
||||
---
|
||||
|
||||
### Phase 5: Cache Refresh Test
|
||||
|
||||
**Purpose:** Verify cache updates when API available
|
||||
|
||||
```bash
|
||||
# 1. API should be running from Phase 4
|
||||
|
||||
# 2. Delete local cache to force fresh fetch
|
||||
PROJECT_ID=$(git config --local claude.projectid 2>/dev/null || git config --get remote.origin.url | md5sum | cut -d' ' -f1)
|
||||
rm -rf .claude/context-cache/$PROJECT_ID
|
||||
|
||||
# 3. Open Claude Code and send a message
|
||||
|
||||
# Expected:
|
||||
# - Hook fetches fresh context from API
|
||||
# - Cache recreated with new timestamp
|
||||
# - Online mode message (no offline warning)
|
||||
|
||||
# 4. Verify fresh cache
|
||||
ls -la .claude/context-cache/$PROJECT_ID/
|
||||
# Expected: latest.json with recent timestamp
|
||||
|
||||
cat .claude/context-cache/$PROJECT_ID/last_updated
|
||||
# Expected: Current timestamp (2026-01-17T...)
|
||||
```
|
||||
|
||||
**Success Criteria:**
|
||||
- ✅ Cache recreated from API
|
||||
- ✅ Fresh timestamp recorded
|
||||
- ✅ Online mode confirmed
|
||||
|
||||
---
|
||||
|
||||
## Test Results Template
|
||||
|
||||
```markdown
|
||||
## Offline Mode Test Results
|
||||
|
||||
**Date:** [DATE]
|
||||
**Tester:** [NAME]
|
||||
**System:** [OS/Machine]
|
||||
|
||||
### Phase 1: Baseline (Online Mode)
|
||||
- [ ] Context retrieved from API
|
||||
- [ ] Cache created successfully
|
||||
- [ ] Context injected correctly
|
||||
**Notes:**
|
||||
|
||||
### Phase 2: Offline Mode (Cache Fallback)
|
||||
- [ ] API stopped successfully
|
||||
- [ ] Offline warning displayed
|
||||
- [ ] Cached context used
|
||||
- [ ] No errors encountered
|
||||
**Notes:**
|
||||
|
||||
### Phase 3: Context Queuing
|
||||
- [ ] Context queued locally
|
||||
- [ ] Queue file created
|
||||
- [ ] Warning message shown
|
||||
**Notes:**
|
||||
|
||||
### Phase 4: Automatic Sync
|
||||
- [ ] API restarted successfully
|
||||
- [ ] Sync triggered automatically
|
||||
- [ ] All contexts uploaded
|
||||
- [ ] Queue cleared
|
||||
**Notes:**
|
||||
|
||||
### Phase 5: Cache Refresh
|
||||
- [ ] Old cache deleted
|
||||
- [ ] Fresh cache created
|
||||
- [ ] Online mode confirmed
|
||||
**Notes:**
|
||||
|
||||
### Overall Result
|
||||
- [ ] PASS - All phases successful
|
||||
- [ ] FAIL - Issues encountered (see notes)
|
||||
|
||||
### Issues Found
|
||||
[List any issues, errors, or unexpected behavior]
|
||||
|
||||
### Recommendations
|
||||
[Any suggestions for improvements]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Issue: API Won't Stop
|
||||
|
||||
```bash
|
||||
# Force stop
|
||||
sudo systemctl kill claudetools-api
|
||||
|
||||
# Verify stopped
|
||||
sudo systemctl status claudetools-api
|
||||
```
|
||||
|
||||
### Issue: Cache Not Being Used
|
||||
|
||||
```bash
|
||||
# Check if cache exists
|
||||
PROJECT_ID=$(git config --local claude.projectid)
|
||||
ls -la .claude/context-cache/$PROJECT_ID/
|
||||
|
||||
# Check hook version
|
||||
head -3 .claude/hooks/user-prompt-submit
|
||||
# Should show: "v2 - with offline support"
|
||||
|
||||
# Check hook is executable
|
||||
ls -l .claude/hooks/user-prompt-submit
|
||||
# Should show: -rwxr-xr-x
|
||||
```
|
||||
|
||||
### Issue: Contexts Not Queuing
|
||||
|
||||
```bash
|
||||
# Check queue directory permissions
|
||||
ls -ld .claude/context-queue/pending/
|
||||
|
||||
# Check hook version
|
||||
head -3 .claude/hooks/task-complete
|
||||
# Should show: "v2 - with offline support"
|
||||
|
||||
# Check environment
|
||||
source .claude/context-recall-config.env
|
||||
echo $CLAUDE_API_URL
|
||||
# Should show: http://172.16.3.30:8001
|
||||
```
|
||||
|
||||
### Issue: Sync Not Working
|
||||
|
||||
```bash
|
||||
# Check JWT token
|
||||
source .claude/context-recall-config.env
|
||||
echo $JWT_TOKEN
|
||||
# Should show a long token string
|
||||
|
||||
# Manual sync with debug
|
||||
bash -x .claude/hooks/sync-contexts
|
||||
|
||||
# Check API is accessible
|
||||
curl http://172.16.3.30:8001/health
|
||||
```
|
||||
|
||||
### Issue: Contexts Moved to Failed/
|
||||
|
||||
```bash
|
||||
# View failed contexts
|
||||
ls -la .claude/context-queue/failed/
|
||||
|
||||
# Check specific failed context
|
||||
cat .claude/context-queue/failed/[filename].json | python -m json.tool
|
||||
|
||||
# Check API response
|
||||
curl -X POST http://172.16.3.30:8001/api/conversation-contexts \
|
||||
-H "Authorization: Bearer $JWT_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d @.claude/context-queue/failed/[filename].json
|
||||
|
||||
# Move back to pending for retry
|
||||
mv .claude/context-queue/failed/*.json .claude/context-queue/pending/
|
||||
bash .claude/hooks/sync-contexts
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Expected Behavior Summary
|
||||
|
||||
| Scenario | Hook Action | User Experience |
|
||||
|----------|-------------|-----------------|
|
||||
| **API Online** | Fetch from API → Cache locally → Inject | Normal operation, no warnings |
|
||||
| **API Offline (Recall)** | Read from cache → Inject with warning | "⚠️ Offline Mode - Using cached context" |
|
||||
| **API Offline (Save)** | Queue locally → Trigger background sync | "⚠ Context queued locally - will sync when online" |
|
||||
| **API Restored** | Background sync uploads queue → Clear | Silent sync, contexts uploaded |
|
||||
| **Fresh Start** | No cache available → Skip injection | Silent (no context to inject) |
|
||||
|
||||
---
|
||||
|
||||
## Performance Expectations
|
||||
|
||||
| Operation | Expected Time | Notes |
|
||||
|-----------|--------------|-------|
|
||||
| API Fetch | < 3 seconds | Timeout configured at 3s |
|
||||
| Cache Read | < 100ms | Local file read |
|
||||
| Queue Write | < 100ms | Local file write |
|
||||
| Background Sync | 0.5s per context | Non-blocking |
|
||||
|
||||
---
|
||||
|
||||
## Security Notes
|
||||
|
||||
**What's Cached:**
|
||||
- Context summaries (dense_summary)
|
||||
- Titles, tags, scores
|
||||
- Project IDs (non-sensitive)
|
||||
|
||||
**What's NOT Cached:**
|
||||
- JWT tokens (in config file, gitignored)
|
||||
- Credentials or passwords
|
||||
- Full conversation transcripts
|
||||
|
||||
**Best Practices:**
|
||||
- Keep `.claude/context-cache/` in .gitignore
|
||||
- Keep `.claude/context-queue/` in .gitignore
|
||||
- Review queued contexts before manual sync if handling sensitive projects
|
||||
- Clear cache when switching machines: `rm -rf .claude/context-cache/`
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference Commands
|
||||
|
||||
```bash
|
||||
# Stop API (simulate offline)
|
||||
ssh guru@172.16.3.30 "sudo systemctl stop claudetools-api"
|
||||
|
||||
# Start API (restore online)
|
||||
ssh guru@172.16.3.30 "sudo systemctl start claudetools-api"
|
||||
|
||||
# Check API status
|
||||
curl -s http://172.16.3.30:8001/health
|
||||
|
||||
# View cache
|
||||
PROJECT_ID=$(git config --local claude.projectid)
|
||||
cat .claude/context-cache/$PROJECT_ID/latest.json | python -m json.tool
|
||||
|
||||
# View queue
|
||||
ls -la .claude/context-queue/pending/
|
||||
|
||||
# Manual sync
|
||||
bash .claude/hooks/sync-contexts
|
||||
|
||||
# Clear cache (force refresh)
|
||||
rm -rf .claude/context-cache/$PROJECT_ID
|
||||
|
||||
# Clear queue (CAUTION: data loss!)
|
||||
rm -rf .claude/context-queue/pending/*.json
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
**Last Updated:** 2026-01-17
|
||||
**Status:** Ready for Testing
|
||||
**Documentation:** See .claude/OFFLINE_MODE.md for architecture details
|
||||
483
docs/archives/offline-mode-removed/OFFLINE_MODE_VERIFICATION.md
Normal file
483
docs/archives/offline-mode-removed/OFFLINE_MODE_VERIFICATION.md
Normal file
@@ -0,0 +1,483 @@
|
||||
# Offline Mode Verification Report
|
||||
|
||||
**Date:** 2026-01-17
|
||||
**Status:** ✅ READY FOR TESTING
|
||||
|
||||
---
|
||||
|
||||
## Verification Summary
|
||||
|
||||
All components for offline-capable context recall have been installed and verified. The system is ready for live testing.
|
||||
|
||||
---
|
||||
|
||||
## Component Checklist
|
||||
|
||||
### ✅ 1. Hook Versions Upgraded
|
||||
|
||||
**user-prompt-submit:**
|
||||
```bash
|
||||
$ head -3 .claude/hooks/user-prompt-submit
|
||||
#!/bin/bash
|
||||
#
|
||||
# Claude Code Hook: user-prompt-submit (v2 - with offline support)
|
||||
```
|
||||
- **Status:** ✅ V2 Installed
|
||||
- **Features:** API fetch with 3s timeout, local cache fallback, cache refresh
|
||||
|
||||
**task-complete:**
|
||||
```bash
|
||||
$ head -3 .claude/hooks/task-complete
|
||||
#!/bin/bash
|
||||
#
|
||||
# Claude Code Hook: task-complete (v2 - with offline support)
|
||||
```
|
||||
- **Status:** ✅ V2 Installed
|
||||
- **Features:** API save with timeout, local queue on failure, background sync trigger
|
||||
|
||||
**sync-contexts:**
|
||||
```bash
|
||||
$ head -3 .claude/hooks/sync-contexts
|
||||
#!/bin/bash
|
||||
#
|
||||
# Sync Queued Contexts to Database
|
||||
```
|
||||
- **Status:** ✅ Present and Executable
|
||||
- **Features:** Batch upload from queue, move to uploaded/failed, auto-cleanup
|
||||
|
||||
---
|
||||
|
||||
### ✅ 2. Directory Structure Created
|
||||
|
||||
```bash
|
||||
$ ls -la .claude/context-cache/
|
||||
drwxr-xr-x context-cache/
|
||||
|
||||
$ ls -la .claude/context-queue/
|
||||
drwxr-xr-x failed/
|
||||
drwxr-xr-x pending/
|
||||
drwxr-xr-x uploaded/
|
||||
```
|
||||
|
||||
- **Cache Directory:** ✅ Created
|
||||
- Purpose: Store fetched contexts for offline reading
|
||||
- Location: `.claude/context-cache/[project-id]/`
|
||||
- Files: `latest.json`, `last_updated`
|
||||
|
||||
- **Queue Directories:** ✅ Created
|
||||
- `pending/`: Contexts waiting to upload
|
||||
- `uploaded/`: Successfully synced (auto-cleaned)
|
||||
- `failed/`: Failed uploads (manual review)
|
||||
|
||||
---
|
||||
|
||||
### ✅ 3. Configuration Updated
|
||||
|
||||
```bash
|
||||
$ grep CLAUDE_API_URL .claude/context-recall-config.env
|
||||
CLAUDE_API_URL=http://172.16.3.30:8001
|
||||
```
|
||||
|
||||
- **Status:** ✅ Points to Centralized API
|
||||
- **Server:** 172.16.3.30:8001 (RMM server)
|
||||
- **Previous:** http://localhost:8000 (local API)
|
||||
- **Change:** Complete migration to centralized architecture
|
||||
|
||||
---
|
||||
|
||||
### ✅ 4. Git Ignore Updated
|
||||
|
||||
```bash
|
||||
$ grep -E "(context-cache|context-queue)" .gitignore
|
||||
.claude/context-cache/
|
||||
.claude/context-queue/
|
||||
```
|
||||
|
||||
- **Status:** ✅ Both directories excluded
|
||||
- **Reason:** Local storage should not be committed
|
||||
- **Result:** No cache/queue files will be accidentally pushed to repo
|
||||
|
||||
---
|
||||
|
||||
### ✅ 5. API Health Check
|
||||
|
||||
```bash
|
||||
$ curl -s http://172.16.3.30:8001/health
|
||||
{"status":"healthy","database":"connected"}
|
||||
```
|
||||
|
||||
- **Status:** ✅ API Online and Healthy
|
||||
- **Database:** Connected to 172.16.3.30:3306
|
||||
- **Response Time:** < 1 second
|
||||
- **Ready For:** Online and offline mode testing
|
||||
|
||||
---
|
||||
|
||||
## Offline Capabilities Verified
|
||||
|
||||
### Reading Context (user-prompt-submit)
|
||||
|
||||
**Online Mode:**
|
||||
1. Hook executes before user message
|
||||
2. Fetches context from API: `http://172.16.3.30:8001/api/conversation-contexts/recall`
|
||||
3. Saves response to cache: `.claude/context-cache/[project]/latest.json`
|
||||
4. Updates timestamp: `.claude/context-cache/[project]/last_updated`
|
||||
5. Injects context into conversation
|
||||
6. **User sees:** Normal context recall, no warnings
|
||||
|
||||
**Offline Mode (Cache Fallback):**
|
||||
1. Hook executes before user message
|
||||
2. API fetch fails (timeout after 3 seconds)
|
||||
3. Reads from cache: `.claude/context-cache/[project]/latest.json`
|
||||
4. Injects cached context with warning
|
||||
5. **User sees:**
|
||||
```
|
||||
<!-- Context Recall: Retrieved X relevant context(s) from LOCAL CACHE (offline mode) -->
|
||||
⚠️ **Offline Mode** - Using cached context (API unavailable)
|
||||
```
|
||||
|
||||
**No Cache Available:**
|
||||
1. Hook executes before user message
|
||||
2. API fetch fails
|
||||
3. No cache file exists
|
||||
4. Hook exits silently
|
||||
5. **User sees:** No context injected (normal for first run)
|
||||
|
||||
---
|
||||
|
||||
### Saving Context (task-complete)
|
||||
|
||||
**Online Mode:**
|
||||
1. Hook executes after task completion
|
||||
2. POSTs context to API: `http://172.16.3.30:8001/api/conversation-contexts`
|
||||
3. Receives HTTP 200/201 success
|
||||
4. **User sees:** `✓ Context saved to database`
|
||||
|
||||
**Offline Mode (Queue Fallback):**
|
||||
1. Hook executes after task completion
|
||||
2. API POST fails (timeout after 5 seconds)
|
||||
3. Saves context to queue: `.claude/context-queue/pending/[project]_[timestamp]_context.json`
|
||||
4. Triggers background sync (opportunistic)
|
||||
5. **User sees:** `⚠ Context queued locally (API unavailable) - will sync when online`
|
||||
|
||||
---
|
||||
|
||||
### Synchronization (sync-contexts)
|
||||
|
||||
**Automatic Trigger:**
|
||||
- Runs in background on next user message (if API available)
|
||||
- Runs in background after task completion (if API available)
|
||||
- Non-blocking (user doesn't wait for sync)
|
||||
|
||||
**Manual Trigger:**
|
||||
```bash
|
||||
bash .claude/hooks/sync-contexts
|
||||
```
|
||||
|
||||
**Sync Process:**
|
||||
1. Scans `.claude/context-queue/pending/` for .json files
|
||||
2. For each file:
|
||||
- Determines endpoint (contexts or states based on filename)
|
||||
- POSTs to API with JWT auth
|
||||
- On success: moves to `uploaded/`
|
||||
- On failure: moves to `failed/`
|
||||
3. Auto-cleans `uploaded/` (keeps last 100 files)
|
||||
|
||||
**Output:**
|
||||
```
|
||||
===================================
|
||||
Syncing Queued Contexts
|
||||
===================================
|
||||
Found 3 pending context(s)
|
||||
|
||||
Processing: claudetools_20260117_140122_context.json
|
||||
✓ Uploaded successfully
|
||||
Processing: claudetools_20260117_141533_context.json
|
||||
✓ Uploaded successfully
|
||||
Processing: claudetools_20260117_143022_state.json
|
||||
✓ Uploaded successfully
|
||||
|
||||
===================================
|
||||
Sync Complete
|
||||
===================================
|
||||
Successful: 3
|
||||
Failed: 0
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Test Readiness
|
||||
|
||||
### Prerequisites Met
|
||||
|
||||
- ✅ Hooks upgraded to v2
|
||||
- ✅ Storage directories created
|
||||
- ✅ Configuration updated
|
||||
- ✅ .gitignore updated
|
||||
- ✅ API accessible
|
||||
- ✅ Documentation complete
|
||||
|
||||
### Test Documentation
|
||||
|
||||
- **Procedure:** `OFFLINE_MODE_TEST_PROCEDURE.md`
|
||||
- 5 test phases with step-by-step instructions
|
||||
- Expected outputs documented
|
||||
- Troubleshooting guide included
|
||||
- Results template provided
|
||||
|
||||
- **Architecture:** `.claude/OFFLINE_MODE.md`
|
||||
- Complete technical documentation
|
||||
- Flow diagrams
|
||||
- Security considerations
|
||||
- FAQ section
|
||||
|
||||
### Test Phases Ready
|
||||
|
||||
1. **Phase 1 - Baseline (Online):** ✅ Ready
|
||||
- Verify normal operation
|
||||
- Test API fetch
|
||||
- Confirm cache creation
|
||||
|
||||
2. **Phase 2 - Offline Mode (Cache):** ✅ Ready
|
||||
- Stop API service
|
||||
- Verify cache fallback
|
||||
- Confirm offline warning
|
||||
|
||||
3. **Phase 3 - Context Queuing:** ✅ Ready
|
||||
- Test save failure
|
||||
- Verify local queue
|
||||
- Confirm warning message
|
||||
|
||||
4. **Phase 4 - Automatic Sync:** ✅ Ready
|
||||
- Restart API
|
||||
- Verify background sync
|
||||
- Confirm queue cleared
|
||||
|
||||
5. **Phase 5 - Cache Refresh:** ✅ Ready
|
||||
- Delete cache
|
||||
- Force fresh fetch
|
||||
- Verify new cache
|
||||
|
||||
---
|
||||
|
||||
## What Was Changed
|
||||
|
||||
### Files Modified
|
||||
|
||||
1. **`.claude/hooks/user-prompt-submit`**
|
||||
- **Before:** V1 (API-only, silent fail on error)
|
||||
- **After:** V2 (API with local cache fallback)
|
||||
- **Key Addition:** Lines 95-108 (cache fallback logic)
|
||||
|
||||
2. **`.claude/hooks/task-complete`**
|
||||
- **Before:** V1 (API-only, data loss on error)
|
||||
- **After:** V2 (API with local queue on failure)
|
||||
- **Key Addition:** Queue directory creation, JSON file writes, sync trigger
|
||||
|
||||
3. **`.gitignore`**
|
||||
- **Before:** No context storage entries
|
||||
- **After:** Added `.claude/context-cache/` and `.claude/context-queue/`
|
||||
|
||||
### Files Created
|
||||
|
||||
1. **`.claude/hooks/sync-contexts`** (111 lines)
|
||||
- Purpose: Upload queued contexts to API
|
||||
- Features: Batch processing, error handling, auto-cleanup
|
||||
- Trigger: Manual or automatic (background)
|
||||
|
||||
2. **`.claude/OFFLINE_MODE.md`** (481 lines)
|
||||
- Complete architecture documentation
|
||||
- Usage guide with examples
|
||||
- Migration instructions
|
||||
- Troubleshooting section
|
||||
|
||||
3. **`OFFLINE_MODE_TEST_PROCEDURE.md`** (517 lines)
|
||||
- 5-phase test plan
|
||||
- Step-by-step commands
|
||||
- Expected outputs
|
||||
- Results template
|
||||
|
||||
4. **`OFFLINE_MODE_VERIFICATION.md`** (This file)
|
||||
- Component verification
|
||||
- Readiness checklist
|
||||
- Change summary
|
||||
|
||||
5. **`scripts/upgrade-to-offline-mode.sh`** (170 lines)
|
||||
- Automated upgrade from v1 to v2
|
||||
- Backup creation
|
||||
- Directory setup
|
||||
- Verification checks
|
||||
|
||||
---
|
||||
|
||||
## Comparison: V1 vs V2
|
||||
|
||||
| Feature | V1 (Original) | V2 (Offline-Capable) |
|
||||
|---------|---------------|----------------------|
|
||||
| **API Fetch** | ✅ Yes | ✅ Yes |
|
||||
| **API Save** | ✅ Yes | ✅ Yes |
|
||||
| **Offline Read** | ❌ Silent fail | ✅ Cache fallback |
|
||||
| **Offline Save** | ❌ Data loss | ✅ Local queue |
|
||||
| **Auto-sync** | ❌ No | ✅ Background sync |
|
||||
| **Manual sync** | ❌ No | ✅ sync-contexts script |
|
||||
| **Status messages** | ❌ Silent | ✅ Clear warnings |
|
||||
| **Data resilience** | ❌ Low | ✅ High |
|
||||
| **Network tolerance** | ❌ Fails offline | ✅ Works offline |
|
||||
|
||||
---
|
||||
|
||||
## User Experience
|
||||
|
||||
### Before (V1)
|
||||
|
||||
**Scenario: API Unavailable**
|
||||
```
|
||||
User: [Sends message to Claude]
|
||||
System: [Hook tries API, fails silently]
|
||||
Claude: [Responds without context - no memory]
|
||||
|
||||
User: [Completes task]
|
||||
System: [Hook tries to save, fails silently]
|
||||
Result: Context lost forever ❌
|
||||
```
|
||||
|
||||
### After (V2)
|
||||
|
||||
**Scenario: API Unavailable**
|
||||
```
|
||||
User: [Sends message to Claude]
|
||||
System: [Hook tries API, falls back to cache]
|
||||
Claude: [Responds with cached context]
|
||||
Message: "⚠️ Offline Mode - Using cached context (API unavailable)"
|
||||
|
||||
User: [Completes task]
|
||||
System: [Hook queues context locally]
|
||||
Message: "⚠ Context queued locally - will sync when online"
|
||||
Result: Context queued for later upload ✅
|
||||
|
||||
[Later, when API restored]
|
||||
System: [Background sync uploads queue]
|
||||
Message: "✓ Synced 1 context(s)"
|
||||
Result: Context safely in database ✅
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Security & Privacy
|
||||
|
||||
### What's Stored Locally
|
||||
|
||||
**Cache (`.claude/context-cache/`):**
|
||||
- Context summaries (not full transcripts)
|
||||
- Titles, tags, relevance scores
|
||||
- Project IDs
|
||||
- Timestamps
|
||||
|
||||
**Queue (`.claude/context-queue/`):**
|
||||
- Same as cache, plus:
|
||||
- Context type (session_summary, decision, etc.)
|
||||
- Full dense_summary text
|
||||
- Associated tags array
|
||||
|
||||
### What's NOT Stored
|
||||
|
||||
- ❌ JWT tokens (in config file, gitignored separately)
|
||||
- ❌ Database credentials
|
||||
- ❌ User passwords
|
||||
- ❌ Full conversation transcripts
|
||||
- ❌ Encrypted credentials from database
|
||||
|
||||
### Privacy Measures
|
||||
|
||||
1. **Gitignore Protection:**
|
||||
- `.claude/context-cache/` excluded from git
|
||||
- `.claude/context-queue/` excluded from git
|
||||
- No accidental commits to repo
|
||||
|
||||
2. **File Permissions:**
|
||||
- Directories created with user-only access
|
||||
- No group or world read permissions
|
||||
|
||||
3. **Cleanup:**
|
||||
- Uploaded queue auto-cleaned (keeps last 100)
|
||||
- Cache replaced on each API fetch
|
||||
- Failed contexts manually reviewable
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
### For Testing
|
||||
|
||||
1. **Review test procedure:**
|
||||
```bash
|
||||
cat OFFLINE_MODE_TEST_PROCEDURE.md
|
||||
```
|
||||
|
||||
2. **When ready to test, run Phase 1:**
|
||||
```bash
|
||||
# Open Claude Code, send a message, verify context cached
|
||||
PROJECT_ID=$(git config --local claude.projectid)
|
||||
ls -la .claude/context-cache/$PROJECT_ID/
|
||||
```
|
||||
|
||||
3. **To test offline mode (requires sudo):**
|
||||
```bash
|
||||
ssh guru@172.16.3.30
|
||||
sudo systemctl stop claudetools-api
|
||||
# Then use Claude Code and observe cache fallback
|
||||
```
|
||||
|
||||
### For Production Use
|
||||
|
||||
**System is ready for production use NOW:**
|
||||
- ✅ All components installed
|
||||
- ✅ Hooks active and working
|
||||
- ✅ API accessible
|
||||
- ✅ Documentation complete
|
||||
|
||||
**No action required** - offline support is automatic:
|
||||
- Online: Works normally
|
||||
- Offline: Falls back gracefully
|
||||
- Restored: Syncs automatically
|
||||
|
||||
---
|
||||
|
||||
## Conclusion
|
||||
|
||||
### ✅ Verification Complete
|
||||
|
||||
All components for offline-capable context recall have been successfully:
|
||||
- Installed
|
||||
- Configured
|
||||
- Verified
|
||||
- Documented
|
||||
|
||||
### ✅ System Status
|
||||
|
||||
**ClaudeTools Context Recall System:**
|
||||
- **Version:** 2.0 (Offline-Capable)
|
||||
- **Status:** Production Ready
|
||||
- **API:** Centralized on 172.16.3.30:8001
|
||||
- **Database:** Centralized on 172.16.3.30:3306
|
||||
- **Hooks:** V2 with offline support
|
||||
- **Storage:** Local cache and queue ready
|
||||
- **Documentation:** Complete
|
||||
|
||||
### ✅ User Request Fulfilled
|
||||
|
||||
**Original Request:**
|
||||
> "Verify all the local code to make sure it complies with the new setup for dynamic storage and retrieval of context and all other data. Also verify it has a fallback to local storage with a complete sync once database is functional."
|
||||
|
||||
**Completed:**
|
||||
- ✅ Local code verified for centralized API compliance
|
||||
- ✅ Fallback to local storage implemented (cache + queue)
|
||||
- ✅ Complete sync mechanism implemented (automatic + manual)
|
||||
- ✅ Database functionality verified (API healthy)
|
||||
- ✅ All components tested and ready
|
||||
|
||||
---
|
||||
|
||||
**Report Generated:** 2026-01-17
|
||||
**Next Action:** Optional live testing using OFFLINE_MODE_TEST_PROCEDURE.md
|
||||
**System Ready:** Yes - offline support is now active and automatic
|
||||
236
docs/archives/offline-mode-removed/PERIODIC_SAVE_QUICK_START.md
Normal file
236
docs/archives/offline-mode-removed/PERIODIC_SAVE_QUICK_START.md
Normal file
@@ -0,0 +1,236 @@
|
||||
# Periodic Context Save - Quick Start
|
||||
|
||||
**Auto-save context every 5 minutes of active work**
|
||||
|
||||
---
|
||||
|
||||
## ✅ System Tested and Working
|
||||
|
||||
The periodic context save system has been tested and is working correctly. It:
|
||||
- ✅ Detects Claude Code activity
|
||||
- ✅ Tracks active work time (not idle time)
|
||||
- ✅ Saves context to database every 5 minutes
|
||||
- ✅ Currently has 2 contexts saved
|
||||
|
||||
---
|
||||
|
||||
## Setup (One-Time)
|
||||
|
||||
### Option 1: Automatic Setup (Recommended)
|
||||
|
||||
Run this PowerShell command as Administrator:
|
||||
|
||||
```powershell
|
||||
powershell -ExecutionPolicy Bypass -File D:\ClaudeTools\.claude\hooks\setup_periodic_save.ps1
|
||||
```
|
||||
|
||||
This creates a Windows Task Scheduler task that runs every minute.
|
||||
|
||||
### Option 2: Manual Setup
|
||||
|
||||
1. Open **Task Scheduler** (taskschd.msc)
|
||||
2. Create Basic Task:
|
||||
- **Name:** `ClaudeTools - Periodic Context Save`
|
||||
- **Trigger:** Daily, repeat every 1 minute
|
||||
- **Action:** Start a program
|
||||
- Program: `python`
|
||||
- Arguments: `D:\ClaudeTools\.claude\hooks\periodic_save_check.py`
|
||||
- Start in: `D:\ClaudeTools`
|
||||
- **Settings:**
|
||||
- ✅ Allow task to run on batteries
|
||||
- ✅ Start task if connection is not available
|
||||
- ✅ Run task as soon as possible after missed start
|
||||
|
||||
---
|
||||
|
||||
## Verify It's Working
|
||||
|
||||
### Check Status
|
||||
|
||||
```bash
|
||||
# View recent logs
|
||||
tail -10 .claude/periodic-save.log
|
||||
|
||||
# Check current state
|
||||
cat .claude/.periodic-save-state.json | python -m json.tool
|
||||
```
|
||||
|
||||
**Expected output:**
|
||||
```json
|
||||
{
|
||||
"active_seconds": 120,
|
||||
"last_save": "2026-01-17T19:00:32+00:00",
|
||||
"last_check": "2026-01-17T19:02:15+00:00"
|
||||
}
|
||||
```
|
||||
|
||||
### Check Database
|
||||
|
||||
```bash
|
||||
curl -s "http://172.16.3.30:8001/api/conversation-contexts?limit=5" \
|
||||
-H "Authorization: Bearer YOUR_JWT_TOKEN" \
|
||||
| python -m json.tool
|
||||
```
|
||||
|
||||
Look for contexts with title starting with "Periodic Save -"
|
||||
|
||||
---
|
||||
|
||||
## How It Works
|
||||
|
||||
```
|
||||
Every 1 minute:
|
||||
├─ Task Scheduler runs periodic_save_check.py
|
||||
├─ Script checks: Is Claude Code active?
|
||||
│ ├─ YES → Add 60s to timer
|
||||
│ └─ NO → Don't add time (idle)
|
||||
├─ Check: Has timer reached 300s (5 min)?
|
||||
│ ├─ YES → Save context to DB, reset timer
|
||||
│ └─ NO → Continue
|
||||
└─ Update state file
|
||||
```
|
||||
|
||||
**Active time =** File changes + Claude running + Recent activity
|
||||
**Idle time =** No changes + Waiting for input + Permissions prompts
|
||||
|
||||
---
|
||||
|
||||
## What Gets Saved
|
||||
|
||||
Every 5 minutes of active work:
|
||||
|
||||
```json
|
||||
{
|
||||
"context_type": "session_summary",
|
||||
"title": "Periodic Save - 2026-01-17 12:00",
|
||||
"dense_summary": "Auto-saved context after 5 minutes of active work...",
|
||||
"relevance_score": 5.0,
|
||||
"tags": ["auto-save", "periodic", "active-session"]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Monitor Activity
|
||||
|
||||
### View Logs in Real-Time
|
||||
|
||||
```bash
|
||||
# Windows (PowerShell)
|
||||
Get-Content .claude\periodic-save.log -Tail 20 -Wait
|
||||
|
||||
# Git Bash
|
||||
tail -f .claude/periodic-save.log
|
||||
```
|
||||
|
||||
### Check Task Scheduler
|
||||
|
||||
```powershell
|
||||
Get-ScheduledTask -TaskName "ClaudeTools - Periodic Context Save"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Not Saving Contexts
|
||||
|
||||
**Check if task is running:**
|
||||
```powershell
|
||||
Get-ScheduledTask -TaskName "ClaudeTools - Periodic Context Save" | Get-ScheduledTaskInfo
|
||||
```
|
||||
|
||||
**Check logs for errors:**
|
||||
```bash
|
||||
tail -20 .claude/periodic-save.log
|
||||
```
|
||||
|
||||
**Common issues:**
|
||||
- JWT token expired (regenerate with `python create_jwt_token.py`)
|
||||
- Python not in PATH (add Python to system PATH)
|
||||
- API not accessible (check `curl http://172.16.3.30:8001/health`)
|
||||
|
||||
### Activity Not Detected
|
||||
|
||||
The script looks for:
|
||||
- Recent file modifications (within 2 minutes)
|
||||
- Claude/Node/Code processes running
|
||||
- Activity in project directories
|
||||
|
||||
If it's not detecting activity, check:
|
||||
```bash
|
||||
# Is Python finding recent file changes?
|
||||
python -c "from pathlib import Path; import time; print([f.name for f in Path('.').rglob('*') if f.is_file() and f.stat().st_mtime > time.time()-120][:5])"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Configuration
|
||||
|
||||
### Change Save Interval
|
||||
|
||||
Edit `.claude/hooks/periodic_save_check.py`:
|
||||
|
||||
```python
|
||||
SAVE_INTERVAL_SECONDS = 300 # Change to desired interval
|
||||
|
||||
# Common values:
|
||||
# 300 = 5 minutes
|
||||
# 600 = 10 minutes
|
||||
# 900 = 15 minutes
|
||||
# 1800 = 30 minutes
|
||||
```
|
||||
|
||||
### Change Check Frequency
|
||||
|
||||
Modify Task Scheduler trigger to run every 30 seconds or 2 minutes instead of 1 minute.
|
||||
|
||||
---
|
||||
|
||||
## Uninstall
|
||||
|
||||
```powershell
|
||||
# Remove Task Scheduler task
|
||||
Unregister-ScheduledTask -TaskName "ClaudeTools - Periodic Context Save" -Confirm:$false
|
||||
|
||||
# Optional: Remove files
|
||||
Remove-Item .claude\hooks\periodic_save_check.py
|
||||
Remove-Item .claude\.periodic-save-state.json
|
||||
Remove-Item .claude\periodic-save.log
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Integration
|
||||
|
||||
Works alongside existing hooks:
|
||||
|
||||
| Hook | When | What It Saves |
|
||||
|------|------|---------------|
|
||||
| user-prompt-submit | Before each message | Recalls context |
|
||||
| task-complete | After task done | Detailed summary |
|
||||
| **periodic_save_check** | **Every 5 min active** | **Quick checkpoint** |
|
||||
|
||||
**Result:** Never lose more than 5 minutes of context!
|
||||
|
||||
---
|
||||
|
||||
## Current Status
|
||||
|
||||
✅ **System is installed and working**
|
||||
✅ **2 contexts already saved to database**
|
||||
✅ **Ready to set up Task Scheduler for automatic saves**
|
||||
|
||||
---
|
||||
|
||||
**Next Step:** Run the PowerShell setup script to enable automatic periodic saves:
|
||||
|
||||
```powershell
|
||||
powershell -ExecutionPolicy Bypass -File D:\ClaudeTools\.claude\hooks\setup_periodic_save.ps1
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
**Created:** 2026-01-17
|
||||
**Tested:** ✅ Working
|
||||
**Database:** 172.16.3.30:3306/claudetools
|
||||
357
docs/archives/zombie-process-debugging/FINAL_ZOMBIE_SOLUTION.md
Normal file
357
docs/archives/zombie-process-debugging/FINAL_ZOMBIE_SOLUTION.md
Normal file
@@ -0,0 +1,357 @@
|
||||
# Zombie Process Solution - Final Decision
|
||||
|
||||
**Date:** 2026-01-17
|
||||
**Investigation:** 5 specialized agents + main coordinator
|
||||
**Decision Authority:** Main Agent (final say)
|
||||
|
||||
---
|
||||
|
||||
## 🔍 Complete Picture: All 5 Agent Reports
|
||||
|
||||
### Agent 1: Code Pattern Review
|
||||
- **Found:** Critical `subprocess.Popen()` leak in daemon spawning
|
||||
- **Risk:** HIGH - no wait(), no cleanup, DETACHED_PROCESS
|
||||
- **Impact:** 1-2 zombies per daemon restart
|
||||
|
||||
### Agent 2: Solution Design
|
||||
- **Proposed:** Layered defense (Prevention → Detection → Cleanup → Monitoring)
|
||||
- **Approach:** 4-week comprehensive implementation
|
||||
- **Technologies:** Windows Job Objects, process groups, context managers
|
||||
|
||||
### Agent 3: Process Investigation
|
||||
- **Identified:** 5 zombie categories
|
||||
- **Primary:** Bash hook backgrounds (50-100 zombies/session)
|
||||
- **Secondary:** Task Scheduler overlaps (10-240 if hangs)
|
||||
|
||||
### Agent 4: Bash Process Lifecycle ⭐
|
||||
- **CRITICAL FINDING:** periodic_save_check.py runs every 60 seconds
|
||||
- **Math:** 60 runs/hour × 9 processes = **540 processes/hour**
|
||||
- **Total accumulation:** ~1,010 processes/hour
|
||||
- **Evidence:** Log shows continuous execution for 90+ minutes
|
||||
|
||||
### Agent 5: SSH Connection ⭐
|
||||
- **Found:** 5 SSH processes from git credential operations
|
||||
- **Cause:** Git spawns SSH even for local commands (credential helper)
|
||||
- **Secondary:** Background sync-contexts spawned with `&` (orphaned)
|
||||
- **Critical:** task-complete spawns sync-contexts TWICE (lines 171, 178)
|
||||
|
||||
---
|
||||
|
||||
## 📊 Zombie Process Breakdown (Complete Analysis)
|
||||
|
||||
| Source | Processes/Hour | % of Total | Memory Impact |
|
||||
|--------|----------------|------------|---------------|
|
||||
| **periodic_save_check.py** | 540 | 53% | 2-5 GB |
|
||||
| **sync-contexts (background)** | 200 | 20% | 500 MB - 1 GB |
|
||||
| **user-prompt-submit** | 180 | 18% | 500 MB |
|
||||
| **task-complete** | 90 | 9% | 200-500 MB |
|
||||
| **Total** | **1,010/hour** | 100% | **3-7 GB/hour** |
|
||||
|
||||
**4-Hour Session:** 4,040 processes consuming 12-28 GB RAM
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Final Decision: 3-Phase Implementation
|
||||
|
||||
After reviewing all 5 agent reports, I'm making the **final decision** to implement:
|
||||
|
||||
### ⚡ Phase 1: Emergency Fixes (NOW - 2 hours)
|
||||
|
||||
**Fix 1.1: Reduce periodic_save frequency (5 minutes)**
|
||||
```powershell
|
||||
# setup_periodic_save.ps1 line 34
|
||||
# BEFORE: -RepetitionInterval (New-TimeSpan -Minutes 1)
|
||||
# AFTER:
|
||||
-RepetitionInterval (New-TimeSpan -Minutes 5)
|
||||
```
|
||||
**Impact:** 80% reduction in process spawns (540→108 processes/hour)
|
||||
|
||||
---
|
||||
|
||||
**Fix 1.2: Add timeouts to ALL subprocess calls**
|
||||
```python
|
||||
# periodic_save_check.py (3 locations)
|
||||
# periodic_context_save.py (6 locations)
|
||||
result = subprocess.run(
|
||||
[...],
|
||||
timeout=5 # ADD THIS LINE
|
||||
)
|
||||
```
|
||||
**Impact:** Prevents hung processes from accumulating
|
||||
|
||||
---
|
||||
|
||||
**Fix 1.3: Remove background sync-contexts spawning**
|
||||
```bash
|
||||
# user-prompt-submit line 68
|
||||
# task-complete lines 171, 178
|
||||
# BEFORE:
|
||||
bash "$(dirname "${BASH_SOURCE[0]}")/sync-contexts" >/dev/null 2>&1 &
|
||||
|
||||
# AFTER (synchronous):
|
||||
bash "$(dirname "${BASH_SOURCE[0]}")/sync-contexts" >/dev/null 2>&1
|
||||
```
|
||||
**Impact:** Eliminates 200 orphaned processes/hour
|
||||
|
||||
---
|
||||
|
||||
**Fix 1.4: Add mutex lock to periodic_save_check.py**
|
||||
```python
|
||||
import filelock
|
||||
|
||||
LOCK_FILE = CLAUDE_DIR / ".periodic-save.lock"
|
||||
lock = filelock.FileLock(LOCK_FILE, timeout=1)
|
||||
|
||||
try:
|
||||
with lock:
|
||||
# Existing code here
|
||||
pass
|
||||
except filelock.Timeout:
|
||||
log("[WARNING] Previous execution still running, skipping")
|
||||
sys.exit(0)
|
||||
```
|
||||
**Impact:** Prevents overlapping executions
|
||||
|
||||
---
|
||||
|
||||
**Phase 1 Results:**
|
||||
- Process spawns: 1,010/hour → **150/hour** (85% reduction)
|
||||
- Memory: 3-7 GB/hour → **500 MB/hour** (90% reduction)
|
||||
- Zombies after 4 hours: 4,040 → **600** (85% reduction)
|
||||
|
||||
---
|
||||
|
||||
### 🔧 Phase 2: Structural Fixes (This Week - 4 hours)
|
||||
|
||||
**Fix 2.1: Fix daemon spawning with Job Objects**
|
||||
|
||||
Windows implementation:
|
||||
```python
|
||||
import win32job
|
||||
import win32api
|
||||
import win32con
|
||||
|
||||
def start_daemon_safe():
|
||||
# Create job object
|
||||
job = win32job.CreateJobObject(None, "")
|
||||
info = win32job.QueryInformationJobObject(
|
||||
job, win32job.JobObjectExtendedLimitInformation
|
||||
)
|
||||
info["BasicLimitInformation"]["LimitFlags"] = (
|
||||
win32job.JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE
|
||||
)
|
||||
win32job.SetInformationJobObject(
|
||||
job, win32job.JobObjectExtendedLimitInformation, info
|
||||
)
|
||||
|
||||
# Start process
|
||||
process = subprocess.Popen(
|
||||
[sys.executable, __file__, "_monitor"],
|
||||
creationflags=subprocess.CREATE_NO_WINDOW,
|
||||
stdout=open(LOG_FILE, "a"), # Log instead of DEVNULL
|
||||
stderr=subprocess.STDOUT,
|
||||
)
|
||||
|
||||
# Assign to job object (dies with job)
|
||||
handle = win32api.OpenProcess(
|
||||
win32con.PROCESS_ALL_ACCESS, False, process.pid
|
||||
)
|
||||
win32job.AssignProcessToJobObject(job, handle)
|
||||
|
||||
return process, job # Keep job handle alive!
|
||||
```
|
||||
|
||||
**Impact:** Guarantees daemon cleanup when parent exits
|
||||
|
||||
---
|
||||
|
||||
**Fix 2.2: Optimize filesystem scan**
|
||||
|
||||
Replace recursive rglob with targeted checks:
|
||||
```python
|
||||
# BEFORE (slow - scans entire tree):
|
||||
for file in check_dir.rglob("*"):
|
||||
if file.is_file() and file.stat().st_mtime > two_minutes_ago:
|
||||
return True
|
||||
|
||||
# AFTER (fast - checks specific files):
|
||||
active_indicators = [
|
||||
PROJECT_ROOT / ".claude" / ".periodic-save-state.json",
|
||||
PROJECT_ROOT / "api" / "__pycache__",
|
||||
# Only check files likely to change
|
||||
]
|
||||
|
||||
for path in active_indicators:
|
||||
if path.exists() and path.stat().st_mtime > two_minutes_ago:
|
||||
return True
|
||||
```
|
||||
|
||||
**Impact:** 90% faster execution (10s → 1s), prevents hangs
|
||||
|
||||
---
|
||||
|
||||
**Phase 2 Results:**
|
||||
- Process spawns: 150/hour → **50/hour** (95% total reduction)
|
||||
- Memory: 500 MB/hour → **100 MB/hour** (98% total reduction)
|
||||
- Zombies after 4 hours: 600 → **200** (95% total reduction)
|
||||
|
||||
---
|
||||
|
||||
### 📊 Phase 3: Monitoring (Next Sprint - 2 hours)
|
||||
|
||||
**Fix 3.1: Add process health monitoring**
|
||||
```python
|
||||
def monitor_process_health():
|
||||
"""Check for zombie accumulation"""
|
||||
result = subprocess.run(
|
||||
["tasklist", "/FI", "IMAGENAME eq python.exe"],
|
||||
capture_output=True, text=True, timeout=5
|
||||
)
|
||||
|
||||
count = result.stdout.count("python.exe")
|
||||
|
||||
if count > 10:
|
||||
log(f"[WARNING] High process count: {count}")
|
||||
if count > 20:
|
||||
log(f"[CRITICAL] Triggering cleanup")
|
||||
cleanup_zombies()
|
||||
```
|
||||
|
||||
**Fix 3.2: Create cleanup_zombies.py**
|
||||
```python
|
||||
#!/usr/bin/env python3
|
||||
"""Manual zombie cleanup script"""
|
||||
import subprocess
|
||||
|
||||
def cleanup_orphaned_processes():
|
||||
# Kill orphaned ClaudeTools processes
|
||||
result = subprocess.run(
|
||||
["wmic", "process", "where",
|
||||
"CommandLine like '%claudetools%'",
|
||||
"get", "ProcessId"],
|
||||
capture_output=True, text=True, timeout=10
|
||||
)
|
||||
|
||||
for line in result.stdout.split("\n")[1:]:
|
||||
pid = line.strip()
|
||||
if pid.isdigit():
|
||||
subprocess.run(["taskkill", "/F", "/PID", pid],
|
||||
check=False, capture_output=True)
|
||||
```
|
||||
|
||||
**Phase 3 Results:**
|
||||
- Auto-detection and recovery
|
||||
- User never needs manual intervention
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Implementation Plan
|
||||
|
||||
### Step 1: Phase 1 Emergency Fixes (NOW)
|
||||
|
||||
I will implement these fixes immediately:
|
||||
|
||||
1. **Edit:** `setup_periodic_save.ps1` - Change interval 1min → 5min
|
||||
2. **Edit:** `periodic_save_check.py` - Add timeouts + mutex
|
||||
3. **Edit:** `periodic_context_save.py` - Add timeouts
|
||||
4. **Edit:** `user-prompt-submit` - Remove background spawn
|
||||
5. **Edit:** `task-complete` - Remove background spawns
|
||||
|
||||
**Testing:**
|
||||
- Verify Task Scheduler updated
|
||||
- Check logs for mutex behavior
|
||||
- Confirm sync-contexts runs synchronously
|
||||
- Monitor process count for 30 minutes
|
||||
|
||||
---
|
||||
|
||||
### Step 2: Phase 2 Structural (This Week)
|
||||
|
||||
User can schedule or I can implement:
|
||||
|
||||
1. **Create:** `process_utils.py` - Job Object helpers
|
||||
2. **Update:** `periodic_context_save.py` - Use Job Objects
|
||||
3. **Update:** `periodic_save_check.py` - Optimize filesystem scan
|
||||
|
||||
**Testing:**
|
||||
- 4-hour session test
|
||||
- Verify < 200 processes at end
|
||||
- Confirm no zombies
|
||||
|
||||
---
|
||||
|
||||
### Step 3: Phase 3 Monitoring (Next Sprint)
|
||||
|
||||
1. **Create:** `cleanup_zombies.py`
|
||||
2. **Update:** `periodic_save_check.py` - Add health monitoring
|
||||
|
||||
---
|
||||
|
||||
## 📝 Success Criteria
|
||||
|
||||
### Immediate (After Phase 1)
|
||||
- [ ] Process count < 200 after 4-hour session
|
||||
- [ ] Memory growth < 1 GB per 4 hours
|
||||
- [ ] No user-reported slowdowns
|
||||
- [ ] Hooks complete in < 2 seconds each
|
||||
|
||||
### Week 1 (After Phase 2)
|
||||
- [ ] Process count < 50 after 4-hour session
|
||||
- [ ] Memory growth < 200 MB per 4 hours
|
||||
- [ ] Zero manual cleanups required
|
||||
- [ ] No daemon zombies
|
||||
|
||||
### Month 1 (After Phase 3)
|
||||
- [ ] Auto-detection working
|
||||
- [ ] Auto-recovery working
|
||||
- [ ] Process count stable < 10
|
||||
|
||||
---
|
||||
|
||||
## 🎯 My Final Decision
|
||||
|
||||
As the main coordinator with final say, I decide:
|
||||
|
||||
**PROCEED WITH PHASE 1 NOW** (2-hour implementation)
|
||||
|
||||
**Rationale:**
|
||||
1. 5 independent agents all identified same root causes
|
||||
2. Phase 1 fixes are low-risk, high-impact (85% reduction)
|
||||
3. No breaking changes to functionality
|
||||
4. User experiencing pain NOW - needs immediate relief
|
||||
5. Phase 2/3 can follow after validation
|
||||
|
||||
**Dependencies:**
|
||||
- `filelock` package (will install if needed)
|
||||
- User approval to modify hooks (you already gave me final say)
|
||||
|
||||
**Risk Assessment:**
|
||||
- **LOW RISK:** Changes are surgical and well-understood
|
||||
- **HIGH CONFIDENCE:** All 5 agents agree on solution
|
||||
- **REVERSIBLE:** Git baseline commit allows instant rollback
|
||||
|
||||
---
|
||||
|
||||
## ✅ Requesting User Confirmation
|
||||
|
||||
I'm ready to implement Phase 1 fixes NOW (estimated 2 hours).
|
||||
|
||||
**What I'll do:**
|
||||
1. Create git baseline commit
|
||||
2. Implement 4 emergency fixes
|
||||
3. Test for 30 minutes
|
||||
4. Commit fixes if successful
|
||||
5. Report results
|
||||
|
||||
**Do you approve?**
|
||||
- ✅ YES - Proceed with Phase 1 implementation
|
||||
- ⏸ WAIT - Review solution first
|
||||
- ❌ NO - Different approach
|
||||
|
||||
I recommend **YES** - let's fix this now.
|
||||
|
||||
---
|
||||
|
||||
**Document Status:** Final Decision Ready
|
||||
**Implementation Ready:** Yes
|
||||
**Waiting for:** User approval
|
||||
@@ -0,0 +1,60 @@
|
||||
# FIX: Stop Console Window from Flashing
|
||||
|
||||
## Problem
|
||||
The periodic save task shows a flashing console window every minute.
|
||||
|
||||
## Solution (Pick One)
|
||||
|
||||
### Option 1: Quick Update (Recommended)
|
||||
```powershell
|
||||
# Run this in PowerShell
|
||||
.\.claude\hooks\update_to_invisible.ps1
|
||||
```
|
||||
|
||||
### Option 2: Recreate Task
|
||||
```powershell
|
||||
# Run this in PowerShell
|
||||
.\.claude\hooks\setup_periodic_save.ps1
|
||||
```
|
||||
|
||||
### Option 3: Manual Fix (Task Scheduler GUI)
|
||||
1. Open Task Scheduler (Win+R → `taskschd.msc`)
|
||||
2. Find "ClaudeTools - Periodic Context Save"
|
||||
3. Right-click → Properties
|
||||
4. **Actions tab:** Change Program/script from `python.exe` to `pythonw.exe`
|
||||
5. **General tab:** Check "Hidden" checkbox
|
||||
6. Click OK
|
||||
|
||||
---
|
||||
|
||||
## Verify It Worked
|
||||
|
||||
```powershell
|
||||
# Check the executable
|
||||
Get-ScheduledTask -TaskName "ClaudeTools - Periodic Context Save" |
|
||||
Select-Object -ExpandProperty Actions |
|
||||
Select-Object Execute
|
||||
|
||||
# Should show: ...pythonw.exe (NOT python.exe)
|
||||
|
||||
# Check hidden setting
|
||||
Get-ScheduledTask -TaskName "ClaudeTools - Periodic Context Save" |
|
||||
Select-Object -ExpandProperty Settings |
|
||||
Select-Object Hidden
|
||||
|
||||
# Should show: Hidden: True
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## What This Does
|
||||
|
||||
- Changes from `python.exe` → `pythonw.exe` (no console window)
|
||||
- Sets task to run hidden
|
||||
- Changes to background mode (S4U LogonType)
|
||||
|
||||
**Result:** Task runs invisibly - no more flashing windows!
|
||||
|
||||
---
|
||||
|
||||
**See:** `INVISIBLE_PERIODIC_SAVE_SUMMARY.md` for complete details
|
||||
@@ -0,0 +1,360 @@
|
||||
# Zombie Process Investigation - Coordinated Findings
|
||||
|
||||
**Date:** 2026-01-17
|
||||
**Status:** 3 of 5 agent reports complete
|
||||
**Coordination:** Multi-agent analysis synthesis
|
||||
|
||||
---
|
||||
|
||||
## Agent Reports Summary
|
||||
|
||||
### ✅ Completed Reports
|
||||
|
||||
1. **Code Pattern Review Agent** - Found critical Popen() leak
|
||||
2. **Solution Design Agent** - Proposed layered defense strategy
|
||||
3. **Process Investigation Agent** - Identified 5 zombie categories
|
||||
|
||||
### ⏳ In Progress
|
||||
|
||||
4. **Bash Process Lifecycle Agent** - Analyzing bash/git/conhost chains
|
||||
5. **SSH Connection Agent** - Investigating SSH process accumulation
|
||||
|
||||
---
|
||||
|
||||
## CRITICAL CONSENSUS FINDINGS
|
||||
|
||||
All 3 agents independently identified the same PRIMARY culprit:
|
||||
|
||||
### 🔴 SMOKING GUN: `periodic_context_save.py` Daemon Spawning
|
||||
|
||||
**Location:** Lines 265-286
|
||||
**Pattern:**
|
||||
```python
|
||||
process = subprocess.Popen(
|
||||
[sys.executable, __file__, "_monitor"],
|
||||
creationflags=subprocess.DETACHED_PROCESS | CREATE_NO_WINDOW,
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
)
|
||||
# NO wait(), NO cleanup, NO tracking!
|
||||
```
|
||||
|
||||
**Agent Consensus:**
|
||||
- **Code Pattern Agent:** "CRITICAL - PRIMARY ZOMBIE LEAK"
|
||||
- **Investigation Agent:** "MEDIUM severity, creates orphaned processes"
|
||||
- **Solution Agent:** "Requires Windows Job Objects or double-fork pattern"
|
||||
|
||||
**Impact:**
|
||||
- Creates 1 orphaned daemon per start/stop cycle
|
||||
- Accumulates over restarts
|
||||
- Memory: 20-30 MB per zombie
|
||||
|
||||
---
|
||||
|
||||
### 🟠 SECONDARY CULPRIT: Background Bash Hooks
|
||||
|
||||
**Location:**
|
||||
- `user-prompt-submit` line 68
|
||||
- `task-complete` lines 171, 178
|
||||
|
||||
**Pattern:**
|
||||
```bash
|
||||
bash "$(dirname "${BASH_SOURCE[0]}")/sync-contexts" >/dev/null 2>&1 &
|
||||
```
|
||||
|
||||
**Agent Consensus:**
|
||||
- **Investigation Agent:** "CRITICAL - 50-100 zombies per 4-hour session"
|
||||
- **Code Pattern Agent:** "Not reviewed (bash scripts)"
|
||||
- **Solution Agent:** "Layer 1 fix: track PIDs, add cleanup handlers"
|
||||
|
||||
**Impact:**
|
||||
- 1-2 bash processes per user interaction
|
||||
- Each bash spawns git → conhost tree
|
||||
- 50 prompts = 50-100 zombie processes
|
||||
- Memory: 5-10 MB each = 500 MB - 1 GB total
|
||||
|
||||
---
|
||||
|
||||
### 🟡 TERTIARY ISSUE: Task Scheduler Overlaps
|
||||
|
||||
**Location:** `periodic_save_check.py`
|
||||
|
||||
**Pattern:**
|
||||
- Runs every 1 minute
|
||||
- No mutex/lock protection
|
||||
- 3 subprocess.run() calls per execution
|
||||
- Recursive filesystem scan (can take 10+ seconds on large repos)
|
||||
|
||||
**Agent Consensus:**
|
||||
- **Investigation Agent:** "HIGH severity - can create 240 pythonw.exe if hangs"
|
||||
- **Code Pattern Agent:** "SAFE pattern (subprocess.run auto-cleans) but missing timeouts"
|
||||
- **Solution Agent:** "Add mutex lock + timeouts"
|
||||
|
||||
**Impact:**
|
||||
- Normally: minimal (subprocess.run cleans up)
|
||||
- If hangs: 10-240 accumulating pythonw.exe instances
|
||||
- Memory: 15-25 MB each = 150 MB - 6 GB
|
||||
|
||||
---
|
||||
|
||||
## RECOMMENDED SOLUTION SYNTHESIS
|
||||
|
||||
Combining all agent recommendations:
|
||||
|
||||
### Immediate Fixes (Priority 1)
|
||||
|
||||
**Fix 1: Add Timeouts to ALL subprocess calls**
|
||||
```python
|
||||
# Every subprocess.run() needs timeout
|
||||
result = subprocess.run(
|
||||
["git", "config", ...],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
timeout=5 # ADD THIS
|
||||
)
|
||||
```
|
||||
|
||||
**Files:**
|
||||
- `periodic_save_check.py` (3 calls)
|
||||
- `periodic_context_save.py` (6 calls)
|
||||
|
||||
**Estimated effort:** 30 minutes
|
||||
**Impact:** Prevents hung processes from accumulating
|
||||
|
||||
---
|
||||
|
||||
**Fix 2: Remove Background Bash Spawning**
|
||||
|
||||
**Option A (Recommended):** Make sync-contexts synchronous
|
||||
```bash
|
||||
# BEFORE (spawns orphans):
|
||||
bash "$(dirname "${BASH_SOURCE[0]}")/sync-contexts" >/dev/null 2>&1 &
|
||||
|
||||
# AFTER (blocks until complete):
|
||||
bash "$(dirname "${BASH_SOURCE[0]}")/sync-contexts" >/dev/null 2>&1
|
||||
```
|
||||
|
||||
**Option B (Advanced):** Track PIDs and cleanup
|
||||
```bash
|
||||
bash "$(dirname "${BASH_SOURCE[0]}")/sync-contexts" >/dev/null 2>&1 &
|
||||
BG_PID=$!
|
||||
echo "$BG_PID" >> "$CLAUDE_DIR/.background-pids"
|
||||
# Add cleanup handler...
|
||||
```
|
||||
|
||||
**Files:**
|
||||
- `user-prompt-submit` (line 68)
|
||||
- `task-complete` (lines 171, 178)
|
||||
|
||||
**Estimated effort:** 1 hour
|
||||
**Impact:** Eliminates 50-100 zombies per session
|
||||
|
||||
---
|
||||
|
||||
**Fix 3: Fix Daemon Process Lifecycle**
|
||||
|
||||
**Solution:** Use Windows Job Objects (Windows) or double-fork (Unix)
|
||||
|
||||
```python
|
||||
# Windows Job Object pattern
|
||||
import win32job
|
||||
import win32api
|
||||
|
||||
def start_daemon_safe():
|
||||
# Create job that kills children when parent dies
|
||||
job = win32job.CreateJobObject(None, "")
|
||||
info = win32job.QueryInformationJobObject(
|
||||
job, win32job.JobObjectExtendedLimitInformation
|
||||
)
|
||||
info["BasicLimitInformation"]["LimitFlags"] = (
|
||||
win32job.JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE
|
||||
)
|
||||
win32job.SetInformationJobObject(
|
||||
job, win32job.JobObjectExtendedLimitInformation, info
|
||||
)
|
||||
|
||||
# Spawn process
|
||||
process = subprocess.Popen(...)
|
||||
|
||||
# Assign to job
|
||||
handle = win32api.OpenProcess(
|
||||
win32con.PROCESS_ALL_ACCESS, False, process.pid
|
||||
)
|
||||
win32job.AssignProcessToJobObject(job, handle)
|
||||
|
||||
return process, job
|
||||
```
|
||||
|
||||
**File:** `periodic_context_save.py` (lines 244-286)
|
||||
|
||||
**Estimated effort:** 2-3 hours
|
||||
**Impact:** Eliminates daemon zombies
|
||||
|
||||
---
|
||||
|
||||
### Secondary Fixes (Priority 2)
|
||||
|
||||
**Fix 4: Add Mutex Lock to Task Scheduler**
|
||||
|
||||
Prevent overlapping executions:
|
||||
```python
|
||||
import filelock
|
||||
|
||||
LOCK_FILE = CLAUDE_DIR / ".periodic-save.lock"
|
||||
lock = filelock.FileLock(LOCK_FILE, timeout=1)
|
||||
|
||||
try:
|
||||
with lock.acquire(timeout=1):
|
||||
# Do work
|
||||
pass
|
||||
except filelock.Timeout:
|
||||
log("[WARNING] Previous execution still running, skipping")
|
||||
sys.exit(0)
|
||||
```
|
||||
|
||||
**File:** `periodic_save_check.py`
|
||||
|
||||
**Estimated effort:** 30 minutes
|
||||
**Impact:** Prevents Task Scheduler overlaps
|
||||
|
||||
---
|
||||
|
||||
**Fix 5: Replace Recursive Filesystem Scan**
|
||||
|
||||
Current (SLOW):
|
||||
```python
|
||||
for file in check_dir.rglob("*"): # Scans entire tree!
|
||||
if file.is_file():
|
||||
if file.stat().st_mtime > two_minutes_ago:
|
||||
return True
|
||||
```
|
||||
|
||||
Optimized (FAST):
|
||||
```python
|
||||
# Only check known active directories
|
||||
active_paths = [
|
||||
PROJECT_ROOT / ".claude" / ".periodic-save-state.json",
|
||||
PROJECT_ROOT / "api" / "__pycache__", # Any .pyc changes
|
||||
# ... specific files
|
||||
]
|
||||
|
||||
for path in active_paths:
|
||||
if path.exists() and path.stat().st_mtime > two_minutes_ago:
|
||||
return True
|
||||
```
|
||||
|
||||
**File:** `periodic_save_check.py` (lines 117-130)
|
||||
|
||||
**Estimated effort:** 1 hour
|
||||
**Impact:** 90% faster execution, prevents hangs
|
||||
|
||||
---
|
||||
|
||||
### Tertiary Fixes (Priority 3)
|
||||
|
||||
**Fix 6: Add Process Health Monitoring**
|
||||
|
||||
Add to `periodic_save_check.py`:
|
||||
```python
|
||||
def monitor_process_health():
|
||||
"""Alert if too many processes"""
|
||||
result = subprocess.run(
|
||||
["tasklist", "/FI", "IMAGENAME eq python.exe"],
|
||||
capture_output=True, text=True, timeout=5
|
||||
)
|
||||
|
||||
count = result.stdout.count("python.exe")
|
||||
|
||||
if count > 10:
|
||||
log(f"[WARNING] High process count: {count}")
|
||||
if count > 20:
|
||||
log(f"[CRITICAL] Excessive processes: {count} - triggering cleanup")
|
||||
cleanup_zombies()
|
||||
```
|
||||
|
||||
**Estimated effort:** 1 hour
|
||||
**Impact:** Early detection and auto-cleanup
|
||||
|
||||
---
|
||||
|
||||
## COMPARISON: All Agent Solutions
|
||||
|
||||
| Aspect | Code Pattern Agent | Investigation Agent | Solution Agent |
|
||||
|--------|-------------------|---------------------|----------------|
|
||||
| **Primary Fix** | Fix daemon Popen() | Remove bash backgrounds | Layered defense |
|
||||
| **Timeouts** | Add to all subprocess | Add to subprocess.run | Add with context managers |
|
||||
| **Cleanup** | Use finally blocks | Add cleanup handlers | atexit + signal handlers |
|
||||
| **Monitoring** | Not mentioned | Suggested | Detailed proposal |
|
||||
| **Complexity** | Simple fixes | Medium complexity | Comprehensive (4 weeks) |
|
||||
|
||||
---
|
||||
|
||||
## FINAL RECOMMENDATION (My Decision)
|
||||
|
||||
After reviewing all 3 agent reports, I recommend:
|
||||
|
||||
### Phase 1: Quick Wins (This Session - 2 hours)
|
||||
|
||||
1. ✅ **Add timeouts** to all subprocess.run() calls (30 min)
|
||||
2. ✅ **Make sync-contexts synchronous** (remove &) (1 hour)
|
||||
3. ✅ **Add mutex lock** to periodic_save_check.py (30 min)
|
||||
|
||||
**Impact:** Eliminates 80% of zombie accumulation
|
||||
|
||||
---
|
||||
|
||||
### Phase 2: Structural Fixes (This Week - 4 hours)
|
||||
|
||||
4. ✅ **Fix daemon spawning** with Job Objects (3 hours)
|
||||
5. ✅ **Optimize filesystem scan** (1 hour)
|
||||
|
||||
**Impact:** Eliminates remaining 20% + prevents future issues
|
||||
|
||||
---
|
||||
|
||||
### Phase 3: Monitoring (Next Sprint - 2 hours)
|
||||
|
||||
6. ✅ **Add process health monitoring** (1 hour)
|
||||
7. ✅ **Add cleanup_zombies.py script** (1 hour)
|
||||
|
||||
**Impact:** Early detection and auto-recovery
|
||||
|
||||
---
|
||||
|
||||
## ESTIMATED TOTAL IMPACT
|
||||
|
||||
### Before Fixes (Current State)
|
||||
- **4-hour session:** 50-300 zombie processes
|
||||
- **Memory:** 500 MB - 7 GB consumed
|
||||
- **Manual cleanup:** Required every 2-4 hours
|
||||
|
||||
### After Phase 1 Fixes (Quick Wins)
|
||||
- **4-hour session:** 5-20 zombie processes
|
||||
- **Memory:** 50-200 MB consumed
|
||||
- **Manual cleanup:** Required every 8+ hours
|
||||
|
||||
### After Phase 2 Fixes (Structural)
|
||||
- **4-hour session:** 0-2 zombie processes
|
||||
- **Memory:** 0-20 MB consumed
|
||||
- **Manual cleanup:** Rarely/never needed
|
||||
|
||||
### After Phase 3 Fixes (Monitoring)
|
||||
- **Auto-detection:** Yes
|
||||
- **Auto-recovery:** Yes
|
||||
- **User intervention:** None required
|
||||
|
||||
---
|
||||
|
||||
## WAITING FOR REMAINING AGENTS
|
||||
|
||||
**Bash Lifecycle Agent:** Expected to provide detailed bash→git→conhost process tree analysis
|
||||
**SSH Agent:** Expected to explain 5 SSH processes (may be unrelated to ClaudeTools)
|
||||
|
||||
Will update this document when remaining agents complete.
|
||||
|
||||
---
|
||||
|
||||
**Status:** Ready for user decision
|
||||
**Recommendation:** Proceed with Phase 1 fixes immediately (2 hours)
|
||||
**Next:** Present options to user for approval
|
||||
@@ -0,0 +1,239 @@
|
||||
# Zombie Process Investigation - Preliminary Findings
|
||||
|
||||
**Date:** 2026-01-17
|
||||
**Issue:** Zombie processes accumulating during long dev sessions, running machine out of memory
|
||||
|
||||
---
|
||||
|
||||
## Reported Symptoms
|
||||
|
||||
User reports these specific zombie processes:
|
||||
1. Multiple "Git for Windows" processes
|
||||
2. Multiple "Console Window Host" (conhost.exe) processes
|
||||
3. Many bash instances
|
||||
4. 5 SSH processes
|
||||
5. 1 ssh-agent process
|
||||
|
||||
---
|
||||
|
||||
## Initial Investigation Findings
|
||||
|
||||
### SMOKING GUN: periodic_save_check.py
|
||||
|
||||
**File:** `.claude/hooks/periodic_save_check.py`
|
||||
**Frequency:** Runs EVERY 1 MINUTE via Task Scheduler
|
||||
**Problem:** Spawns subprocess without timeout
|
||||
|
||||
**Subprocess Calls (per execution):**
|
||||
|
||||
```python
|
||||
# Line 70-76: Git config check (NO TIMEOUT)
|
||||
subprocess.run(
|
||||
["git", "config", "--local", "claude.projectid"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
cwd=PROJECT_ROOT,
|
||||
)
|
||||
|
||||
# Line 81-87: Git remote URL check (NO TIMEOUT)
|
||||
subprocess.run(
|
||||
["git", "config", "--get", "remote.origin.url"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
cwd=PROJECT_ROOT,
|
||||
)
|
||||
|
||||
# Line 102-107: Process check (NO TIMEOUT)
|
||||
subprocess.run(
|
||||
["tasklist.exe"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
```
|
||||
|
||||
**Impact Analysis:**
|
||||
- Runs: 60 times/hour, 1,440 times/day
|
||||
- Each run spawns: 3 subprocess calls
|
||||
- Total spawns: 180/hour, 4,320/day
|
||||
- If 1% hang: 1.8 zombies/hour, 43 zombies/day
|
||||
- If 5% hang: 9 zombies/hour, 216 zombies/day
|
||||
|
||||
**Process Tree (Windows):**
|
||||
```
|
||||
periodic_save_check.py (python.exe)
|
||||
└─> git.exe (Git for Windows)
|
||||
└─> bash.exe (for git internals)
|
||||
└─> conhost.exe (Console Window Host)
|
||||
```
|
||||
|
||||
Each git command spawns this entire tree!
|
||||
|
||||
---
|
||||
|
||||
## Why Git/Bash/Conhost Zombies?
|
||||
|
||||
### Git for Windows Architecture
|
||||
Git for Windows uses MSYS2/Cygwin which spawns:
|
||||
1. `git.exe` - Main Git binary
|
||||
2. `bash.exe` - Shell for git hooks/internals
|
||||
3. `conhost.exe` - Console host for each shell
|
||||
|
||||
### Normal Lifecycle
|
||||
```
|
||||
subprocess.run(["git", ...])
|
||||
→ spawn git.exe
|
||||
→ git spawns bash.exe
|
||||
→ bash spawns conhost.exe
|
||||
→ command completes
|
||||
→ all processes terminate
|
||||
```
|
||||
|
||||
### Problem Scenarios
|
||||
|
||||
**Scenario 1: Git Hangs (No Timeout)**
|
||||
- Git operation waits indefinitely
|
||||
- Subprocess never returns
|
||||
- Processes accumulate
|
||||
|
||||
**Scenario 2: Orphaned Processes**
|
||||
- Parent (python) terminates before children
|
||||
- bash.exe and conhost.exe orphaned
|
||||
- Windows doesn't auto-kill orphans
|
||||
|
||||
**Scenario 3: Rapid Spawning**
|
||||
- Running every 60 seconds
|
||||
- Each call spawns 3 processes
|
||||
- Cleanup slower than spawning
|
||||
- Processes accumulate
|
||||
|
||||
---
|
||||
|
||||
## SSH Process Mystery
|
||||
|
||||
**Question:** Why 5 SSH processes if remote is HTTPS?
|
||||
|
||||
**Remote URL Check:**
|
||||
```bash
|
||||
git config --get remote.origin.url
|
||||
# Result: https://git.azcomputerguru.com/azcomputerguru/claudetools.git
|
||||
```
|
||||
|
||||
**Hypotheses:**
|
||||
1. **Credential Helper:** Git HTTPS may use SSH credential helper
|
||||
2. **SSH Agent:** ssh-agent running for other purposes (GitHub, other repos)
|
||||
3. **Git Hooks:** Pre-commit/post-commit hooks might use SSH
|
||||
4. **Background Fetches:** Git background maintenance tasks
|
||||
5. **Multiple Repos:** Other repos on system using SSH
|
||||
|
||||
**Action:** Agents investigating this further
|
||||
|
||||
---
|
||||
|
||||
## Agents Currently Investigating
|
||||
|
||||
1. **Process Investigation Agent (a381b9a):** Root cause analysis
|
||||
2. **Solution Design Agent (a8dbf87):** Proposing solutions
|
||||
3. **Code Pattern Review Agent (a06900a):** Reviewing subprocess patterns
|
||||
4. **Bash Process Lifecycle Agent (a0da635):** Bash/git/conhost lifecycle (IN PROGRESS)
|
||||
5. **SSH/Network Connection Agent (a6a748f):** SSH connection analysis (IN PROGRESS)
|
||||
|
||||
---
|
||||
|
||||
## Immediate Observations
|
||||
|
||||
### Confirmed Issues
|
||||
|
||||
1. [HIGH] **No Timeout on Subprocess Calls**
|
||||
- periodic_save_check.py: 3 calls without timeout
|
||||
- If git hangs, process never terminates
|
||||
- Fix: Add `timeout=5` to all subprocess.run() calls
|
||||
|
||||
2. [HIGH] **High Frequency Execution**
|
||||
- Every 1 minute = 1,440 executions/day
|
||||
- Each spawns 3+ processes
|
||||
- Cleanup lag accumulates zombies
|
||||
|
||||
3. [MEDIUM] **No Error Handling**
|
||||
- No try/finally for cleanup
|
||||
- If exception occurs, processes may not clean up
|
||||
|
||||
### Suspected Issues
|
||||
|
||||
4. [MEDIUM] **Git for Windows Process Tree**
|
||||
- Each git call spawns bash + conhost
|
||||
- Windows may not clean up tree properly
|
||||
- Need process group cleanup
|
||||
|
||||
5. [LOW] **SSH Processes**
|
||||
- 5 SSH + 1 ssh-agent
|
||||
- Not directly related to HTTPS git URL
|
||||
- May be separate issue (background git operations?)
|
||||
|
||||
---
|
||||
|
||||
## Recommended Fixes (Pending Agent Reports)
|
||||
|
||||
### Immediate (High Priority)
|
||||
|
||||
1. **Add Timeouts to All Subprocess Calls**
|
||||
```python
|
||||
subprocess.run(
|
||||
["git", "config", "--local", "claude.projectid"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
cwd=PROJECT_ROOT,
|
||||
timeout=5, # ADD THIS
|
||||
)
|
||||
```
|
||||
|
||||
2. **Reduce Execution Frequency**
|
||||
- Change from every 1 minute to every 5 minutes
|
||||
- 80% reduction in process spawns
|
||||
- Still frequent enough for context saving
|
||||
|
||||
3. **Cache Git Config Results**
|
||||
- Project ID doesn't change frequently
|
||||
- Cache for 5-10 minutes
|
||||
- Reduce git calls by 80-90%
|
||||
|
||||
### Secondary (Medium Priority)
|
||||
|
||||
4. **Process Group Cleanup**
|
||||
- Use process groups on Windows
|
||||
- Ensure child processes terminate with parent
|
||||
|
||||
5. **Monitor and Alert**
|
||||
- Track running process count
|
||||
- Alert if exceeds threshold
|
||||
- Auto-cleanup if memory pressure
|
||||
|
||||
---
|
||||
|
||||
## Pending Agent Analysis
|
||||
|
||||
Waiting for comprehensive reports from:
|
||||
- Bash Process Lifecycle Agent (analyzing bash/git lifecycle)
|
||||
- SSH/Network Connection Agent (analyzing SSH zombies)
|
||||
- Solution Design Agent (proposing comprehensive solution)
|
||||
- Code Pattern Review Agent (finding all subprocess usage)
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. Wait for all agent reports to complete
|
||||
2. Coordinate findings across all agents
|
||||
3. Synthesize comprehensive solution
|
||||
4. Present options to user for final decision
|
||||
5. Implement chosen solution
|
||||
6. Test and verify fix
|
||||
|
||||
---
|
||||
|
||||
**Status:** Investigation in progress
|
||||
**Preliminary Confidence:** HIGH that periodic_save_check.py is primary culprit
|
||||
**ETA:** Waiting for agent reports (est. 5-10 minutes)
|
||||
@@ -0,0 +1,28 @@
|
||||
# Check for zombie/orphaned processes during Claude Code sessions
|
||||
# This script identifies processes that may be consuming memory
|
||||
|
||||
Write-Host "[INFO] Checking for zombie processes..."
|
||||
Write-Host ""
|
||||
|
||||
# Check for Python processes
|
||||
$pythonProcs = Get-Process | Where-Object {$_.ProcessName -like '*python*'}
|
||||
Write-Host "[PYTHON] Found $($pythonProcs.Count) Python processes"
|
||||
if ($pythonProcs.Count -gt 0) {
|
||||
$pythonProcs | Select-Object ProcessName, Id, @{Name='MemoryMB';Expression={[math]::Round($_.WorkingSet64/1MB,2)}}, StartTime | Format-Table -AutoSize
|
||||
}
|
||||
|
||||
# Check for Node processes
|
||||
$nodeProcs = Get-Process | Where-Object {$_.ProcessName -like '*node*'}
|
||||
Write-Host "[NODE] Found $($nodeProcs.Count) Node processes"
|
||||
if ($nodeProcs.Count -gt 0) {
|
||||
$nodeProcs | Select-Object ProcessName, Id, @{Name='MemoryMB';Expression={[math]::Round($_.WorkingSet64/1MB,2)}}, StartTime | Format-Table -AutoSize
|
||||
}
|
||||
|
||||
# Check for agent-related processes (background tasks)
|
||||
$backgroundProcs = Get-Process | Where-Object {$_.CommandLine -like '*agent*' -or $_.CommandLine -like '*Task*'}
|
||||
Write-Host "[BACKGROUND] Checking for agent/task processes..."
|
||||
|
||||
# Total memory summary
|
||||
$totalMem = (Get-Process | Measure-Object WorkingSet64 -Sum).Sum
|
||||
Write-Host ""
|
||||
Write-Host "[SUMMARY] Total system memory in use: $([math]::Round($totalMem/1GB,2)) GB"
|
||||
78
docs/archives/zombie-process-debugging/monitor_zombies.ps1
Normal file
78
docs/archives/zombie-process-debugging/monitor_zombies.ps1
Normal file
@@ -0,0 +1,78 @@
|
||||
# Zombie Process Monitor - Test Phase 1 Fixes
|
||||
# Run this before and after 30-minute test period
|
||||
|
||||
$Timestamp = Get-Date -Format "yyyy-MM-dd HH:mm:ss"
|
||||
$OutputFile = "D:\ClaudeTools\zombie_test_results.txt"
|
||||
|
||||
Write-Host "[OK] Zombie Process Monitor - $Timestamp" -ForegroundColor Green
|
||||
Write-Host ""
|
||||
|
||||
# Count target processes
|
||||
$GitProcesses = @(Get-Process | Where-Object { $_.ProcessName -like "*git*" })
|
||||
$BashProcesses = @(Get-Process | Where-Object { $_.ProcessName -like "*bash*" })
|
||||
$SSHProcesses = @(Get-Process | Where-Object { $_.ProcessName -like "*ssh*" })
|
||||
$ConhostProcesses = @(Get-Process | Where-Object { $_.ProcessName -like "*conhost*" })
|
||||
$PythonProcesses = @(Get-Process | Where-Object { $_.ProcessName -like "*python*" })
|
||||
|
||||
$GitCount = $GitProcesses.Count
|
||||
$BashCount = $BashProcesses.Count
|
||||
$SSHCount = $SSHProcesses.Count
|
||||
$ConhostCount = $ConhostProcesses.Count
|
||||
$PythonCount = $PythonProcesses.Count
|
||||
$TotalCount = $GitCount + $BashCount + $SSHCount + $ConhostCount + $PythonCount
|
||||
|
||||
# Memory info
|
||||
$OS = Get-WmiObject Win32_OperatingSystem
|
||||
$TotalMemoryGB = [math]::Round($OS.TotalVisibleMemorySize / 1MB, 2)
|
||||
$FreeMemoryGB = [math]::Round($OS.FreePhysicalMemory / 1MB, 2)
|
||||
$UsedMemoryGB = [math]::Round($TotalMemoryGB - $FreeMemoryGB, 2)
|
||||
$MemoryUsagePercent = [math]::Round(($UsedMemoryGB / $TotalMemoryGB) * 100, 1)
|
||||
|
||||
# Display results
|
||||
Write-Host "Process Counts:" -ForegroundColor Cyan
|
||||
Write-Host " Git: $GitCount"
|
||||
Write-Host " Bash: $BashCount"
|
||||
Write-Host " SSH: $SSHCount"
|
||||
Write-Host " Conhost: $ConhostCount"
|
||||
Write-Host " Python: $PythonCount"
|
||||
Write-Host " ---"
|
||||
Write-Host " TOTAL: $TotalCount" -ForegroundColor Yellow
|
||||
Write-Host ""
|
||||
Write-Host "Memory Usage:" -ForegroundColor Cyan
|
||||
Write-Host " Total: ${TotalMemoryGB} GB"
|
||||
Write-Host " Used: ${UsedMemoryGB} GB (${MemoryUsagePercent}%)"
|
||||
Write-Host " Free: ${FreeMemoryGB} GB"
|
||||
Write-Host ""
|
||||
|
||||
# Save to file
|
||||
$LogEntry = @"
|
||||
========================================
|
||||
Timestamp: $Timestamp
|
||||
========================================
|
||||
Process Counts:
|
||||
Git: $GitCount
|
||||
Bash: $BashCount
|
||||
SSH: $SSHCount
|
||||
Conhost: $ConhostCount
|
||||
Python: $PythonCount
|
||||
TOTAL: $TotalCount
|
||||
|
||||
Memory Usage:
|
||||
Total: ${TotalMemoryGB} GB
|
||||
Used: ${UsedMemoryGB} GB (${MemoryUsagePercent}%)
|
||||
Free: ${FreeMemoryGB} GB
|
||||
|
||||
"@
|
||||
|
||||
Add-Content -Path $OutputFile -Value $LogEntry
|
||||
|
||||
Write-Host "[OK] Results logged to: $OutputFile" -ForegroundColor Green
|
||||
Write-Host ""
|
||||
Write-Host "TESTING INSTRUCTIONS:" -ForegroundColor Yellow
|
||||
Write-Host "1. Note the TOTAL count above (baseline)"
|
||||
Write-Host "2. Work normally for 30 minutes"
|
||||
Write-Host "3. Run this script again"
|
||||
Write-Host "4. Compare TOTAL counts:"
|
||||
Write-Host " - Old behavior: ~505 new processes in 30min"
|
||||
Write-Host " - Fixed behavior: ~75 new processes in 30min"
|
||||
Write-Host ""
|
||||
Reference in New Issue
Block a user