Initial commit: ClaudeTools system foundation
Complete architecture for multi-mode Claude operation: - MSP Mode (client work tracking) - Development Mode (project management) - Normal Mode (general research) Agents created: - Coding Agent (perfectionist programmer) - Code Review Agent (quality gatekeeper) - Database Agent (data custodian) - Gitea Agent (version control) - Backup Agent (data protection) Workflows documented: - CODE_WORKFLOW.md (mandatory review process) - TASK_MANAGEMENT.md (checklist system) - FILE_ORGANIZATION.md (hybrid storage) - MSP-MODE-SPEC.md (complete architecture, 36 tables) Commands: - /sync (pull latest from Gitea) Database schema: 36 tables for comprehensive context storage File organization: clients/, projects/, normal/, backups/ Backup strategy: Daily/weekly/monthly with retention Status: Architecture complete, ready for implementation Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
601
.claude/FILE_ORGANIZATION.md
Normal file
601
.claude/FILE_ORGANIZATION.md
Normal file
@@ -0,0 +1,601 @@
|
||||
# File Organization & Storage Strategy
|
||||
|
||||
## Overview
|
||||
|
||||
The ClaudeTools system uses a **hybrid storage approach**:
|
||||
- **Database** - Metadata, context, task status, relationships, insights
|
||||
- **Filesystem** - Actual code files, documentation, scripts, configurations
|
||||
- **Gitea** - Version control for all file-based work
|
||||
- **Backups** - Local database dumps and file backups
|
||||
|
||||
## Storage Philosophy
|
||||
|
||||
### What Goes in Database (via Database Agent)
|
||||
- Task and session metadata
|
||||
- Work item records (problem/solution summaries)
|
||||
- Client and infrastructure details
|
||||
- Environmental insights
|
||||
- Failure patterns
|
||||
- Command history
|
||||
- Credentials (encrypted)
|
||||
- Relationships between entities
|
||||
- Timestamps and audit trails
|
||||
|
||||
### What Goes on Filesystem (version controlled)
|
||||
- Source code files
|
||||
- Configuration files
|
||||
- Documentation (markdown, txt)
|
||||
- Scripts (PowerShell, Bash, Python)
|
||||
- Diagrams and images
|
||||
- README files
|
||||
- Project-specific notes
|
||||
- Session logs (markdown)
|
||||
|
||||
### Why Hybrid?
|
||||
- **Database**: Fast queries, relationships, metadata
|
||||
- **Filesystem**: Version control, diff tracking, editor-friendly
|
||||
- **Gitea**: Backup, history, team collaboration
|
||||
- **Best of both worlds**: Structured data + version-controlled files
|
||||
|
||||
## Folder Structure
|
||||
|
||||
```
|
||||
D:\ClaudeTools\
|
||||
├── .claude\ # System configuration
|
||||
│ ├── agents\ # Agent definitions
|
||||
│ │ ├── coding.md
|
||||
│ │ ├── code-review.md
|
||||
│ │ ├── database.md
|
||||
│ │ ├── gitea.md
|
||||
│ │ └── backup.md
|
||||
│ ├── plans\ # Plan mode outputs
|
||||
│ ├── settings.local.json # Local settings
|
||||
│ ├── CODE_WORKFLOW.md
|
||||
│ ├── TASK_MANAGEMENT.md
|
||||
│ ├── FILE_ORGANIZATION.md
|
||||
│ └── MSP-MODE-SPEC.md
|
||||
│
|
||||
├── clients\ # MSP Mode - Client work
|
||||
│ ├── dataforth\
|
||||
│ │ ├── configs\ # Configuration files
|
||||
│ │ │ ├── nas\
|
||||
│ │ │ │ ├── smb.conf.overrides
|
||||
│ │ │ │ └── sync-to-ad2.sh
|
||||
│ │ │ ├── ad2\
|
||||
│ │ │ └── dos-machines\
|
||||
│ │ ├── docs\ # Documentation
|
||||
│ │ │ ├── README.md
|
||||
│ │ │ ├── NETWORK_TOPOLOGY.md
|
||||
│ │ │ ├── CREDENTIALS.md (encrypted)
|
||||
│ │ │ └── TROUBLESHOOTING.md
|
||||
│ │ ├── scripts\ # Automation scripts
|
||||
│ │ │ ├── Sync-FromNAS.ps1
|
||||
│ │ │ ├── UPDATE.BAT
|
||||
│ │ │ └── monitor-nas.ps1
|
||||
│ │ ├── session-logs\ # Session logs for this client
|
||||
│ │ │ ├── 2026-01-15-dos-update.md
|
||||
│ │ │ └── 2026-01-14-wins-fix.md
|
||||
│ │ ├── .git\ # Git repository
|
||||
│ │ ├── .gitignore
|
||||
│ │ └── README.md # Client overview
|
||||
│ │
|
||||
│ ├── grabb\
|
||||
│ │ ├── configs\
|
||||
│ │ ├── docs\
|
||||
│ │ ├── scripts\
|
||||
│ │ ├── session-logs\
|
||||
│ │ └── README.md
|
||||
│ │
|
||||
│ └── [other-clients]\
|
||||
│
|
||||
├── projects\ # Development Mode - Dev projects
|
||||
│ ├── gururmm\
|
||||
│ │ ├── agent\ # Agent source code (Rust)
|
||||
│ │ │ ├── src\
|
||||
│ │ │ ├── Cargo.toml
|
||||
│ │ │ └── README.md
|
||||
│ │ ├── server\ # Server source code (Rust)
|
||||
│ │ │ ├── src\
|
||||
│ │ │ ├── Cargo.toml
|
||||
│ │ │ └── README.md
|
||||
│ │ ├── dashboard\ # Dashboard source code (React)
|
||||
│ │ │ ├── src\
|
||||
│ │ │ ├── package.json
|
||||
│ │ │ └── README.md
|
||||
│ │ ├── docs\ # Project documentation
|
||||
│ │ │ ├── ARCHITECTURE.md
|
||||
│ │ │ ├── API.md
|
||||
│ │ │ └── DEPLOYMENT.md
|
||||
│ │ ├── scripts\ # Build/deploy scripts
|
||||
│ │ ├── session-logs\ # Development session logs
|
||||
│ │ ├── .git\
|
||||
│ │ └── README.md
|
||||
│ │
|
||||
│ ├── claudetools-api\ # ClaudeTools API project
|
||||
│ │ ├── src\
|
||||
│ │ ├── docs\
|
||||
│ │ ├── tests\
|
||||
│ │ ├── .git\
|
||||
│ │ └── README.md
|
||||
│ │
|
||||
│ └── [other-projects]\
|
||||
│
|
||||
├── normal\ # Normal Mode - General work
|
||||
│ ├── research\ # Research notes
|
||||
│ │ ├── rust-async-patterns.md
|
||||
│ │ ├── docker-networking.md
|
||||
│ │ └── ...
|
||||
│ ├── experiments\ # Code experiments
|
||||
│ │ ├── test-project-1\
|
||||
│ │ └── ...
|
||||
│ ├── notes\ # General notes
|
||||
│ └── session-logs\ # Normal mode session logs
|
||||
│
|
||||
├── backups\ # Backup storage
|
||||
│ ├── database\ # Database backups
|
||||
│ │ ├── claudetools-2026-01-15-daily.sql.gz
|
||||
│ │ ├── claudetools-2026-01-14-daily.sql.gz
|
||||
│ │ └── ...
|
||||
│ ├── files\ # File backups (snapshots)
|
||||
│ │ ├── clients-2026-01-15.tar.gz
|
||||
│ │ └── ...
|
||||
│ └── .gitignore # Don't commit backups
|
||||
│
|
||||
└── README.md # ClaudeTools overview
|
||||
```
|
||||
|
||||
## Gitea Integration
|
||||
|
||||
### Repository Structure
|
||||
|
||||
Each client and project has its own Git repository:
|
||||
|
||||
**Client Repositories:**
|
||||
- `azcomputerguru/claudetools-client-dataforth`
|
||||
- `azcomputerguru/claudetools-client-grabb`
|
||||
- etc.
|
||||
|
||||
**Project Repositories:**
|
||||
- `azcomputerguru/gururmm` (already exists)
|
||||
- `azcomputerguru/claudetools-api`
|
||||
- etc.
|
||||
|
||||
**System Repository:**
|
||||
- `azcomputerguru/claudetools` (for .claude configs, agents, system docs)
|
||||
|
||||
### Commit Strategy
|
||||
|
||||
**When to Commit:**
|
||||
1. **After completing tasks** - Task marked 'completed' in database
|
||||
2. **After significant work** - Code review approved, feature complete
|
||||
3. **End of session** - Session ending, preserve work
|
||||
4. **On user request** - User explicitly asks to commit
|
||||
5. **Periodic commits** - Every N tasks completed (configurable)
|
||||
|
||||
**Commit Message Format:**
|
||||
```
|
||||
[Mode] Brief description
|
||||
|
||||
Detailed context:
|
||||
- Task: [task title]
|
||||
- Changes: [list of changes]
|
||||
- Duration: [time spent]
|
||||
- Status: [completed/in-progress]
|
||||
|
||||
Files modified:
|
||||
- path/to/file1.py
|
||||
- path/to/file2.md
|
||||
```
|
||||
|
||||
**Example Commit:**
|
||||
```
|
||||
[MSP:Dataforth] Fix WINS service on D2TESTNAS
|
||||
|
||||
Detailed context:
|
||||
- Task: Troubleshoot WINS service failure
|
||||
- Changes: Fixed smb.conf.overrides syntax error
|
||||
- Duration: 45 minutes (billable)
|
||||
- Status: Completed and verified
|
||||
|
||||
Files modified:
|
||||
- configs/nas/smb.conf.overrides
|
||||
- docs/TROUBLESHOOTING.md
|
||||
- session-logs/2026-01-15-wins-fix.md
|
||||
|
||||
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
|
||||
```
|
||||
|
||||
### Git Workflow
|
||||
|
||||
**Orchestrator triggers Git operations via Gitea Agent:**
|
||||
|
||||
1. **Detect changes** - Files modified during work
|
||||
2. **Stage changes** - `git add` modified files
|
||||
3. **Create commit** - With context-rich message
|
||||
4. **Push to Gitea** - Sync to remote repository
|
||||
5. **Update database** - Record commit hash in session
|
||||
|
||||
## File Creation Rules
|
||||
|
||||
### Client Files (MSP Mode)
|
||||
|
||||
When working on client project:
|
||||
|
||||
1. **Determine client** - From session context
|
||||
2. **Create/update files** in `D:\ClaudeTools\clients\[client-name]\`
|
||||
3. **Use appropriate subdirectory**:
|
||||
- `configs/` - Configuration files
|
||||
- `docs/` - Documentation
|
||||
- `scripts/` - Automation scripts
|
||||
- `session-logs/` - Session logs
|
||||
4. **Update README.md** - Keep client overview current
|
||||
5. **Stage for commit** - Gitea Agent handles commit
|
||||
|
||||
### Project Files (Development Mode)
|
||||
|
||||
When working on development project:
|
||||
|
||||
1. **Determine project** - From session context
|
||||
2. **Create/update files** in `D:\ClaudeTools\projects\[project-name]\`
|
||||
3. **Follow project structure** - Respect existing organization
|
||||
4. **Update documentation** - Keep docs current
|
||||
5. **Write tests** - Test files alongside code
|
||||
6. **Stage for commit** - Gitea Agent handles commit
|
||||
|
||||
### Normal Mode Files
|
||||
|
||||
When in Normal Mode:
|
||||
|
||||
1. **Categorize work**:
|
||||
- Research → `normal/research/`
|
||||
- Experiments → `normal/experiments/`
|
||||
- Notes → `normal/notes/`
|
||||
2. **Use descriptive filenames** - Include date if temporal
|
||||
3. **Commit periodically** - Keep work safe
|
||||
|
||||
## Database Backup Strategy
|
||||
|
||||
### Backup Schedule
|
||||
|
||||
**Daily backups:**
|
||||
- Automatic at 2:00 AM (if system running)
|
||||
- Or on first session of the day if not run overnight
|
||||
|
||||
**On-demand backups:**
|
||||
- Before major changes (schema updates)
|
||||
- After significant data entry
|
||||
- On user request
|
||||
|
||||
### Backup Format
|
||||
|
||||
**Filename:** `claudetools-YYYY-MM-DD-[type].sql.gz`
|
||||
- `type` = daily, manual, pre-migration, etc.
|
||||
|
||||
**Contents:**
|
||||
- Full database dump (all tables, all data)
|
||||
- Compressed with gzip
|
||||
- Includes schema and data
|
||||
- Stored in `D:\ClaudeTools\backups\database\`
|
||||
|
||||
**Example:**
|
||||
```
|
||||
claudetools-2026-01-15-daily.sql.gz
|
||||
claudetools-2026-01-15-pre-migration.sql.gz
|
||||
claudetools-2026-01-14-daily.sql.gz
|
||||
```
|
||||
|
||||
### Backup Retention
|
||||
|
||||
- **Keep last 7 daily backups**
|
||||
- **Keep last 4 weekly backups** (Sunday backups)
|
||||
- **Keep last 12 monthly backups** (1st of month backups)
|
||||
- **Keep pre-migration backups indefinitely**
|
||||
- **Automatic cleanup** - Backup Agent purges old backups
|
||||
|
||||
### Backup Verification
|
||||
|
||||
After each backup:
|
||||
1. **Check file size** - Not zero, reasonable size
|
||||
2. **Test gzip integrity** - `gzip -t file.sql.gz`
|
||||
3. **Record in database** - Log backup in `backup_log` table
|
||||
4. **Optional: Test restore** - Periodically restore to verify
|
||||
|
||||
## Session Logs
|
||||
|
||||
### Where They Go
|
||||
|
||||
**MSP Mode:**
|
||||
- `D:\ClaudeTools\clients\[client]\session-logs\YYYY-MM-DD-[description].md`
|
||||
|
||||
**Development Mode:**
|
||||
- `D:\ClaudeTools\projects\[project]\session-logs\YYYY-MM-DD-[description].md`
|
||||
|
||||
**Normal Mode:**
|
||||
- `D:\ClaudeTools\normal\session-logs\YYYY-MM-DD-[description].md`
|
||||
|
||||
**Shared (cross-mode sessions):**
|
||||
- `C:\Users\MikeSwanson\claude-projects\session-logs\YYYY-MM-DD-session.md` (existing location)
|
||||
- Symlinks or cross-references in mode-specific locations
|
||||
|
||||
### Session Log Format
|
||||
|
||||
```markdown
|
||||
# Session: [Title]
|
||||
|
||||
**Date:** 2026-01-15
|
||||
**Mode:** MSP (Dataforth) / Development (GuruRMM) / Normal
|
||||
**Duration:** 2.5 hours
|
||||
**Billable:** Yes (MSP only)
|
||||
|
||||
## Summary
|
||||
|
||||
[Brief description of what was accomplished]
|
||||
|
||||
## Tasks Completed
|
||||
|
||||
- [✓] Task 1 (30 min)
|
||||
- [✓] Task 2 (45 min)
|
||||
- [✓] Task 3 (15 min)
|
||||
|
||||
## Work Items (MSP Mode)
|
||||
|
||||
### Problem
|
||||
[Description]
|
||||
|
||||
### Cause
|
||||
[Root cause]
|
||||
|
||||
### Solution
|
||||
[What was done]
|
||||
|
||||
### Verification
|
||||
[How it was tested]
|
||||
|
||||
## Files Modified
|
||||
|
||||
- `path/to/file1.py` - [what changed]
|
||||
- `path/to/file2.md` - [what changed]
|
||||
|
||||
## Key Learnings
|
||||
|
||||
- [Environmental insights discovered]
|
||||
- [Failures encountered and resolved]
|
||||
- [Patterns identified]
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [ ] Pending task 1
|
||||
- [ ] Pending task 2
|
||||
|
||||
## References
|
||||
|
||||
- Related sessions: [links]
|
||||
- Documentation: [links]
|
||||
- Tickets: [ticket IDs if MSP mode]
|
||||
```
|
||||
|
||||
## File Naming Conventions
|
||||
|
||||
### Session Logs
|
||||
`YYYY-MM-DD-brief-description.md`
|
||||
- Example: `2026-01-15-wins-service-fix.md`
|
||||
- Example: `2026-01-14-update-bat-simplification.md`
|
||||
|
||||
### Configuration Files
|
||||
`service-config.ext` or `SERVICE.CONF`
|
||||
- Example: `smb.conf.overrides`
|
||||
- Example: `sync-to-ad2.sh`
|
||||
- Match existing conventions where applicable
|
||||
|
||||
### Scripts
|
||||
`verb-noun.ext`
|
||||
- Example: `Sync-FromNAS.ps1` (PowerShell PascalCase)
|
||||
- Example: `monitor-nas.sh` (Bash kebab-case)
|
||||
- Example: `deploy-app.py` (Python snake_case)
|
||||
|
||||
### Documentation
|
||||
`SCREAMING_CASE.md` for important docs, `Title Case.md` for regular
|
||||
- Example: `README.md`
|
||||
- Example: `ARCHITECTURE.md`
|
||||
- Example: `API Documentation.md`
|
||||
|
||||
## .gitignore Files
|
||||
|
||||
Each repository should have appropriate `.gitignore`:
|
||||
|
||||
**Client repositories:**
|
||||
```gitignore
|
||||
# Credentials (encrypted versions are OK, plaintext never)
|
||||
**/CREDENTIALS.txt
|
||||
**/*password*
|
||||
**/*secret*
|
||||
|
||||
# Temporary files
|
||||
*.tmp
|
||||
*.log
|
||||
*.bak
|
||||
|
||||
# OS files
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
```
|
||||
|
||||
**Project repositories:**
|
||||
```gitignore
|
||||
# Dependencies
|
||||
node_modules/
|
||||
target/
|
||||
venv/
|
||||
__pycache__/
|
||||
|
||||
# Build outputs
|
||||
dist/
|
||||
build/
|
||||
*.exe
|
||||
*.dll
|
||||
|
||||
# IDE
|
||||
.vscode/
|
||||
.idea/
|
||||
*.swp
|
||||
|
||||
# Environment
|
||||
.env
|
||||
.env.local
|
||||
```
|
||||
|
||||
**ClaudeTools root:**
|
||||
```gitignore
|
||||
# Backups (local only)
|
||||
backups/
|
||||
|
||||
# Local settings
|
||||
.claude/settings.local.json
|
||||
|
||||
# Temporary
|
||||
*.tmp
|
||||
```
|
||||
|
||||
## Integration with Database
|
||||
|
||||
### File References in Database
|
||||
|
||||
Store file paths in database for context:
|
||||
|
||||
```python
|
||||
# In task_context field:
|
||||
{
|
||||
"files_modified": [
|
||||
"clients/dataforth/configs/nas/smb.conf.overrides",
|
||||
"clients/dataforth/docs/TROUBLESHOOTING.md"
|
||||
],
|
||||
"files_created": [
|
||||
"clients/dataforth/scripts/monitor-nas.ps1"
|
||||
],
|
||||
"commit_hash": "a3f5b92c...",
|
||||
"repository": "claudetools-client-dataforth"
|
||||
}
|
||||
```
|
||||
|
||||
### Backup References
|
||||
|
||||
Track backups in database:
|
||||
|
||||
```sql
|
||||
CREATE TABLE backup_log (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
backup_type VARCHAR(50) CHECK(backup_type IN ('daily', 'manual', 'pre-migration', 'weekly', 'monthly')),
|
||||
file_path VARCHAR(500) NOT NULL,
|
||||
file_size_bytes BIGINT NOT NULL,
|
||||
backup_started_at TIMESTAMP NOT NULL,
|
||||
backup_completed_at TIMESTAMP NOT NULL,
|
||||
verification_status VARCHAR(50) CHECK(verification_status IN ('passed', 'failed', 'not_verified')),
|
||||
verification_details TEXT,
|
||||
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||
|
||||
INDEX idx_backup_type (backup_type),
|
||||
INDEX idx_backup_date (backup_completed_at)
|
||||
);
|
||||
```
|
||||
|
||||
## Orchestrator Workflow
|
||||
|
||||
### During Work Session
|
||||
|
||||
1. **Session starts** → Detect mode, client/project
|
||||
2. **Create files** → In appropriate directory structure
|
||||
3. **Update database** → Record task context, file paths
|
||||
4. **Complete tasks** → Track what files were modified
|
||||
5. **Periodic commits** → Gitea Agent commits changes
|
||||
6. **Session ends** → Final commit, session log saved
|
||||
|
||||
### Example Flow
|
||||
|
||||
```python
|
||||
# User: "Fix the WINS service on D2TESTNAS"
|
||||
|
||||
# Orchestrator:
|
||||
# 1. Determine context
|
||||
mode = "MSP"
|
||||
client = "Dataforth"
|
||||
base_path = "D:/ClaudeTools/clients/dataforth/"
|
||||
|
||||
# 2. Launch agents to do work
|
||||
# ... Coding Agent fixes config file ...
|
||||
|
||||
# 3. Files created/modified
|
||||
files = [
|
||||
"configs/nas/smb.conf.overrides",
|
||||
"docs/TROUBLESHOOTING.md"
|
||||
]
|
||||
|
||||
# 4. Update database via Database Agent
|
||||
Database_Agent.update_task(
|
||||
task_id=task_id,
|
||||
status="completed",
|
||||
task_context={
|
||||
"files_modified": files,
|
||||
"solution": "Fixed syntax error in smb.conf.overrides",
|
||||
"verification": "WINS service restarted successfully"
|
||||
}
|
||||
)
|
||||
|
||||
# 5. Commit to Gitea via Gitea Agent
|
||||
Gitea_Agent.commit(
|
||||
repository="claudetools-client-dataforth",
|
||||
files=files,
|
||||
message=generate_commit_message(task_context),
|
||||
base_path=base_path
|
||||
)
|
||||
|
||||
# 6. Create session log
|
||||
session_log_path = f"{base_path}session-logs/2026-01-15-wins-fix.md"
|
||||
create_session_log(session_log_path, session_data)
|
||||
|
||||
# 7. Commit session log
|
||||
Gitea_Agent.commit(
|
||||
repository="claudetools-client-dataforth",
|
||||
files=["session-logs/2026-01-15-wins-fix.md"],
|
||||
message="Session log: WINS service fix"
|
||||
)
|
||||
```
|
||||
|
||||
## Benefits
|
||||
|
||||
### For User
|
||||
- **Organized** - Files logically grouped by client/project
|
||||
- **Version controlled** - Complete history in Gitea
|
||||
- **Backed up** - Database and files protected
|
||||
- **Searchable** - Both database queries and file searches
|
||||
- **Recoverable** - Multiple backup layers
|
||||
|
||||
### For System
|
||||
- **Hybrid storage** - Right tool for right data
|
||||
- **Context preservation** - Database + files = complete picture
|
||||
- **Audit trail** - Git history + database records
|
||||
- **Disaster recovery** - Multiple backup strategies
|
||||
- **Scalable** - Clean separation of concerns
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
**Hybrid Approach:**
|
||||
- Database → Metadata, relationships, context, queries
|
||||
- Filesystem → Code, docs, configs (version controlled)
|
||||
- Gitea → Backup, history, collaboration
|
||||
- Local backups → Safety net for database
|
||||
|
||||
**Organization:**
|
||||
- `clients/` → MSP Mode work
|
||||
- `projects/` → Development Mode work
|
||||
- `normal/` → Normal Mode work
|
||||
- `backups/` → Database and file backups
|
||||
|
||||
**Automation:**
|
||||
- Gitea Agent → Periodic commits
|
||||
- Backup Agent → Daily database backups
|
||||
- Database Agent → Track file references
|
||||
- Orchestrator → Coordinate all agents
|
||||
|
||||
**Result:** Complete, organized, version-controlled, backed-up system.
|
||||
Reference in New Issue
Block a user