# Autocoder 2.0 Enhancement Spec - Conversational Format  Use this as your response during the spec creation interview. Copy/paste or adapt as needed.  ---  ## Initial Project Description  I want to enhance Autocoder 2.0 - this is an existing, fully functional autonomous coding agent system. It already works and I use it regularly. I'm looking to add new capabilities that will make it exceptional.  **Current stack:** Python/FastAPI backend, React/TypeScript frontend with Tailwind v4, SQLite database. But I'm open to changes if you think something would work better - feel free to suggest alternatives or refactor if it makes the enhancements cleaner.  ---  ## What Already Exists and Works  **Core Agent System:** - Two-agent pattern: Initializer creates features from app spec, Coding agent implements them - Full agent loop with pause/resume/stop controls - YOLO mode for rapid prototyping (skips browser testing) - Auto-continuation between sessions with 3-second delay  **Feature Management:** - SQLite database with priority, category, name, description, steps, status - MCP server exposing 7 tools (get_stats, get_next, mark_passing, etc.) - Kanban board UI showing pending/in-progress/done  **Parallel Execution:** - `parallel_agent.py` - Run multiple agents on non-conflicting features - `smart_orchestrator.py` - AI-driven parallelization decisions - `feature_graph.py` - Tracks file conflicts between features - `file_lock_manager.py` - Async-safe locking for concurrent access  **Infrastructure Awareness:** - `credential_resolver.py` - Parses credentials.md for server access - `resource_manager.py` - Tracks servers/services with SSH verification - `resource_context.py` - Merges global + project resources - Context injection into agent prompts (agents know about my infrastructure)  **UI & Real-time:** - React 18 with TanStack Query, Radix UI components - Neobrutalism design system with custom animations - WebSocket for real-time logs, status, and feature updates - Project registry supporting projects in any directory - Folder browser for project selection - Multi-step project creation wizard with spec interview  **Server Architecture:** - FastAPI with 77 API routes across 11 routers - Localhost-only security middleware - Services: AgentRunner, SessionManager, SpecGenerator, SmartScheduler, etc.  **Recent Bug Fix (Already Applied):** - Process tree termination - stopping an agent now kills all child processes (MCP servers, Playwright) - Added `/api/system/orphans` and `/api/system/cleanup` endpoints for zombie cleanup - Uses `psutil.Process.children(recursive=True)` for thorough cleanup  ---  ## What I Want to Add  ### Priority 1: Observability & Reliability  **1. Token and Cost Tracking** I want visibility into how much each session and feature costs. Store usage data in SQLite, show running cost in the UI header as a badge that updates in real-time. Include: - Tokens per session (input/output breakdown) - Cost estimate using current Anthropic pricing - Historical data - "this project has used X tokens / $Y total" - Per-feature cost attribution  **2. Session Recovery / Checkpointing** If the agent crashes or I close it accidentally, resume where it left off instead of restarting. Save state periodically - every few tool calls or after significant actions. Options: - Add `--resume` flag to CLI - Or make it automatic - detect incomplete session and offer to resume - Store checkpoint in project directory (`.agent_checkpoint.json` or similar)  **3. Better Error Recovery** When the agent gets stuck or fails repeatedly on a feature: - Detect the stuck state (same error 3+ times) - Option to escalate to a more capable model (Opus) - Option to skip and move to next feature - Log detailed failure info for later analysis  ### Priority 2: Developer Experience  **4. Git Integration** Automate version control during development: - Auto-commit after each feature passes with meaningful message derived from feature description - Option to create branch per feature (`feature/feature-name`) - Generate PR descriptions when ready to merge - Handle merge conflicts with AI assistance (or at least flag them clearly)  **5. Multi-Model Support** Route tasks to appropriate models: - **Opus** - Complex architectural decisions, when stuck, analyzing large codebases - **Sonnet** - Standard coding work (current default) - **Haiku** - Quick validations, linting decisions, simple checks, summarization  Implementation ideas: - Config file for model routing rules - Or let the orchestrator decide dynamically based on task complexity - Fallback chain: try Haiku first, escalate if it struggles  **6. Improved Logging & Debugging** - Structured logs with levels (debug/info/warn/error) - Log viewer in UI with filtering - Export logs for a session - "Why did it do that?" - trace reasoning for specific actions  ### Priority 3: Intelligence & Learning  **7. Learning from Past Work** Build project knowledge over time: - When a feature passes, index the implementation approach - On similar future features, retrieve relevant past solutions - Could be simple (keyword search over saved implementations) or sophisticated (embeddings) - Project-specific knowledge base that grows over time  **8. Smarter Feature Prioritization** Go beyond simple priority numbers: - Detect dependencies automatically from code analysis - Suggest optimal ordering based on complexity and risk - Flag features that might conflict or have shared dependencies - "This feature should come after X because it depends on Y"  ### Priority 4: Visual & UX Improvements  **9. Visual Dashboard** - Dependency graph showing how features relate (D3.js or similar) - Token usage charts over time - Timeline view of agent actions - "Thinking" indicator showing what agent is considering - Cost breakdown by feature/session  **10. Enhanced Agent Controls** - "Retry with Opus" button for stuck features - Queue management - reorder pending features via drag-drop - Bulk operations - skip multiple, reprioritize - Estimated time/cost for remaining work  ---  ## Architecture Considerations  **Plugin/Extension System (Future)** For extensibility, consider: - Pre/post implementation hooks - Custom MCP tools that can be added per-project - Custom validators for project-specific requirements - Would make Autocoder adaptable to different workflows  **Cost Optimization** - Per-project token budgets with warnings - Prompt optimization - don't repeat unchanged context - Cache common operations - Usage analytics and trend detection  ---  ## Guidelines for Implementation  **Don't break what's working** - This is enhancement, not rewrite. The current system works well.  **You can refactor if needed** - If an enhancement would be cleaner with restructuring, go for it. Just ensure existing functionality still works.  **Each feature should be independently testable** - No features that require 3 other things to verify.  **Suggest the tech stack** - If you think Redis would be better for caching, or we need a vector DB for learning, suggest it. I trust your judgment.  **Follow existing patterns** - Look at how `process_manager.py`, `resource_manager.py`, or the MCP server are structured. Consistent style matters.  **Consider the parallel execution model** - New features should work correctly when multiple agents run simultaneously.  ---  ## Constraints (Don't Break These)  - Security model must remain intact - don't bypass bash allowlist in `security.py` - Database migrations if changing schema - no destructive changes to `features.db` - UI changes need `npm run build` afterward - Keep localhost-only restriction for API (security middleware in `main.py`) - Don't commit credentials or API keys - Process cleanup must work - don't spawn processes that can become orphans  ---  ## Testing Strategy  Each feature should be verifiable by: 1. Lint passes (ruff) 2. Type check passes (pyright/mypy if configured) 3. Manual verification of described behavior 4. Doesn't break existing features (run agent on a test project) 5. Process cleanup still works (no zombies)  ---  ## File Locations Reference  For context when implementing:  ``` Autocoder-2.0/ ├── agent.py                 # Main agent loop - ADD checkpointing here ├── client.py                # Claude SDK config - ADD multi-model here ├── progress.py              # Progress tracking - ADD token counting here ├── parallel_agent.py        # Parallel runner ├── smart_orchestrator.py    # AI parallelization │ ├── server/ │   ├── main.py              # FastAPI app (77 routes) │   ├── services/ │   │   ├── process_manager.py  # Process lifecycle (UPDATED with tree kill) │   │   ├── credential_resolver.py │   │   ├── feature_graph.py │   │   ├── smart_scheduler.py  # ADD smarter prioritization here │   │   └── ... │   └── routers/ │       ├── system.py        # NEW - orphan cleanup endpoints │       └── ... │ ├── ui/src/ │   ├── App.tsx              # ADD cost badge, enhanced controls │   ├── components/          # 24 components │   └── hooks/               # ADD useUsage, useCheckpoint hooks │ ├── api/database.py          # SQLAlchemy models - ADD Usage table └── mcp_server/feature_mcp.py  # MCP tools ```  ---  ## Notes for Interview Clarifications  **On priority:** Token tracking and session recovery are highest - they help me immediately. Git integration and multi-model are medium priority. Learning and visual improvements are nice-to-have.  **On scope:** Start with core functionality, don't gold-plate. Working basics beat elaborate features that don't quite work.  **On architecture:** Prefer composition over inheritance, small focused modules, clear separation. The existing services are good examples.  **On the learning feature:** Start simple - even just saving successful implementations as searchable markdown would be valuable. Don't need vector embeddings immediately.  **On multi-model:** Config-driven is fine initially. Dynamic routing can come later.


I'll help you enhance Autocoder 2.0 based on this comprehensive spec. This is an impressive autonomous coding system, and I can see clear paths to make it even better.

Let me start by exploring the current codebase structure to understand the existing architecture and patterns.


Task

Explore Autocoder 2.0 codebase

Prompt:
Explore the Autocoder 2.0 codebase to understand:

1. Current project structure and key files
2. Database schema in api/database.py
3. Existing services architecture in server/services/
4. MCP server implementation
5. Agent loop structure in agent.py
6. UI components and hooks structure
7. Progress tracking and session management

Focus on understanding the existing patterns so I can implement enhancements that follow the same architectural style. Look for:
- How services are structured and initialized
- Database model patterns
- API route organization
- WebSocket implementation for real-time updates
- Process management and cleanup patterns
- Configuration management

This will help me design enhancements that integrate cleanly with the existing system.


Perfect! Now I have comprehensive understanding. Let me create a detailed summary document:

## Autocoder 2.0 Architecture Exploration - Complete Summary

I've completed a thorough exploration of the Autocoder 2.0 codebase. Here's a comprehensive breakdown of the system architecture and patterns:

---

## 1. PROJECT STRUCTURE & KEY FILES

**Root Level:**
- `/c/Users/MikeSwanson/claude-projects/Autocode-remix/Autocoder-2.0/agent.py` - Main agent session loop
- `/c/Users/MikeSwanson/claude-projects/Autocode-remix/Autocoder-2.0/progress.py` - Progress tracking and database queries
- `/c/Users/MikeSwanson/claude-projects/Autocode-remix/Autocoder-2.0/config.py` - Centralized configuration management
- `/c/Users/MikeSwanson/claude-projects/Autocode-remix/Autocoder-2.0/client.py` - Claude SDK client configuration
- `/c/Users/MikeSwanson/claude-projects/Autocode-remix/Autocoder-2.0/registry.py` - Project registry management

**API Layer:**
- `/c/Users/MikeSwanson/claude-projects/Autocode-remix/Autocoder-2.0/api/database.py` - SQLAlchemy ORM models and database functions

**MCP Server:**
- `/c/Users/MikeSwanson/claude-projects/Autocode-remix/Autocoder-2.0/mcp_server/feature_mcp.py` - FastMCP server for feature management

**Server (FastAPI):**
- `/c/Users/MikeSwanson/claude-projects/Autocode-remix/Autocoder-2.0/server/main.py` - FastAPI application entry point
- `/c/Users/MikeSwanson/claude-projects/Autocode-remix/Autocoder-2.0/server/websocket.py` - WebSocket handlers with connection manager
- `/c/Users/MikeSwanson/claude-projects/Autocode-remix/Autocoder-2.0/server/schemas.py` - Pydantic request/response models

**Services:**
- `/c/Users/MikeSwanson/claude-projects/Autocode-remix/Autocoder-2.0/server/services/process_manager.py` - Agent subprocess lifecycle management
- `/c/Users/MikeSwanson/claude-projects/Autocode-remix/Autocoder-2.0/server/services/resource_manager.py` - Infrastructure resource management
- `/c/Users/MikeSwanson/claude-projects/Autocode-remix/Autocoder-2.0/server/services/feature_graph.py` - Feature dependency graph tracking
- `/c/Users/MikeSwanson/claude-projects/Autocode-remix/Autocoder-2.0/server/services/smart_scheduler.py` - Intelligent parallelization scheduling
- `/c/Users/MikeSwanson/claude-projects/Autocode-remix/Autocoder-2.0/server/services/terminal_session.py` - Interactive terminal for resource management
- `/c/Users/MikeSwanson/claude-projects/Autocode-remix/Autocoder-2.0/server/services/credential_resolver.py` - Infrastructure credential parsing

**Routers:**
- `/c/Users/MikeSwanson/claude-projects/Autocode-remix/Autocoder-2.0/server/routers/projects.py` - Project CRUD endpoints
- `/c/Users/MikeSwanson/claude-projects/Autocode-remix/Autocoder-2.0/server/routers/features.py` - Feature management endpoints
- `/c/Users/MikeSwanson/claude-projects/Autocode-remix/Autocoder-2.0/server/routers/agent.py` - Agent control endpoints

**UI:**
- `/c/Users/MikeSwanson/claude-projects/Autocode-remix/Autocoder-2.0/ui/src/hooks/` - React Query hooks for data fetching
- `/c/Users/MikeSwanson/claude-projects/Autocode-remix/Autocoder-2.0/ui/src/components/` - 24+ UI components

---

## 2. DATABASE SCHEMA (api/database.py)

**Feature Model:**
```python
class Feature(Base):
    __tablename__ = "features"

    id: int (primary key)
    priority: int (default=999, indexed)      # Lower = higher priority
    category: str(100)                        # Feature category
    name: str(255)                            # Feature name
    description: str                          # Detailed description
    steps: JSON array                         # Test/implementation steps
    passes: bool (default=False, indexed)    # Completion status
    in_progress: bool (default=False, indexed) # Currently being worked on
```

**Database Initialization:**
- Path: `{project_dir}/features.db` (SQLite)
- Automatic schema creation via `create_database()` function
- Migration support for adding `in_progress` column to legacy databases
- Cross-platform path handling using POSIX-style URLs

**Session Management:**
```python
def get_database_path(project_dir: Path) -> Path
def get_database_url(project_dir: Path) -> str  # Returns: sqlite:///{posix_path}
def create_database(project_dir: Path) -> tuple  # Returns: (engine, SessionLocal)
def set_session_maker(session_maker: sessionmaker) -> None  # FastAPI dependency
def get_db() -> Session  # FastAPI dependency injection
```

---

## 3. SERVICES ARCHITECTURE (server/services/)

### Pattern: Singleton Manager Pattern with Thread Safety

All services follow a consistent pattern:

```python
# Global manager with thread safety
_manager_instance = None
_manager_lock = threading.Lock()

def get_manager(...) -> ServiceManager:
    """Get or create singleton instance (thread-safe)"""
    with _manager_lock:
        if _manager_instance is None:
            _manager_instance = ServiceManager(...)
        return _manager_instance
```

### A. Process Manager (process_manager.py)

**Responsibilities:**
- Subprocess lifecycle (start/stop/pause/resume)
- Cross-platform support (Windows via psutil)
- Process output streaming to WebSocket clients
- Lock file management to prevent multiple instances
- Sensitive data redaction in logs

**Key Classes:**
```python
class AgentProcessManager:
    def __init__(project_name: str, project_dir: Path, root_dir: Path)

    # Properties
    @property status: Literal["stopped", "running", "paused", "crashed"]
    @property pid: int | None
    @property started_at: datetime | None

    # Methods
    async def start(yolo_mode: bool = False) -> tuple[bool, str]
    async def stop() -> tuple[bool, str]
    async def pause() -> tuple[bool, str]
    async def resume() -> tuple[bool, str]
    async def healthcheck() -> bool

    # Callbacks (multiple listeners supported)
    def add_output_callback(callback: Callable[[str], Awaitable[None]])
    def add_status_callback(callback: Callable[[str], Awaitable[None]])

    # Internal
    def _check_lock() -> bool
    def _create_lock() -> None
    def _remove_lock() -> None
    async def _stream_output() -> None
    async def _broadcast_output(line: str) -> None
```

**Output Sanitization:**
- Redacts API keys, tokens, passwords, AWS credentials
- Regex patterns for Anthropic, GitHub, AWS keys

**Process Isolation:**
- Lock file: `{project_dir}/.agent.lock` (contains PID)
- Prevents multiple agent instances per project
- Orphan detection via `psutil.pid_exists()`

### B. Resource Manager (resource_manager.py)

**Responsibilities:**
- Track servers, services, databases, repositories
- Infrastructure-aware context injection
- Credential resolution with SSH/API auth support
- Resource verification and capability probing

**Key Models:**
```python
@dataclass
class Resource:
    name: str
    type: ResourceType  # SERVER, SERVICE, DATABASE, REPOSITORY
    scope: ResourceScope  # GLOBAL, PROJECT
    host: str | None
    port: int | None
    url: str | None
    user: str | None
    auth: ResourceAuth
    capabilities: list[str]  # Probed at verification time
    verified_at: str | None
    last_error: str | None

@dataclass
class ResourceAuth:
    type: AuthType  # SSH_KEY, SSH_PASSWORD, API_TOKEN, API_KEY, BASIC_AUTH
    credential_ref: str | None  # Reference to credentials.md
```

**Manager Methods:**
```python
class ResourceManager:
    def add_resource(resource: Resource, scope: ResourceScope) -> Resource
    def update_resource(name: str, updates: dict) -> Resource
    def get_resource(name: str) -> Optional[Resource]
    def list_resources(scope: ResourceScope | None = None) -> list[Resource]
    def verify_resource(name: str) -> VerificationResult
    def probe_capabilities(resource: Resource) -> list[str]
    def get_context_for_agent() -> str  # Returns formatted context
```

### C. Feature Graph (feature_graph.py)

**Responsibilities:**
- Track feature dependencies and file conflicts
- Enable parallel execution analysis
- Separate database: `{project_dir}/feature_graph.db`

**Key Classes:**
```python
@dataclass
class ParallelFeature:
    id: int
    priority: int
    category: str
    name: str
    description: str
    steps: list[str]
    estimated_complexity: int  # 1-5 scale

@dataclass
class FeatureConflict:
    feature_id: int
    conflicting_feature_id: int
    reason: str
    conflicting_files: list[str]

class FeatureGraph:
    def __init__(project_dir: Path)

    # Analysis
    def detect_conflicts(feature_ids: list[int]) -> list[FeatureConflict]
    def find_parallelizable_groups(features: list[Feature]) -> list[list[int]]
    def get_estimated_complexity(feature_id: int) -> int

    # File tracking
    def track_files_touched(feature_id: int, files: list[str])
    def get_files_for_feature(feature_id: int) -> list[str]

    # Locking
    def acquire_file_locks(files: list[str], feature_id: int) -> bool
    def release_file_locks(feature_id: int) -> None
```

### D. Smart Scheduler (smart_scheduler.py)

**Responsibilities:**
- AI-driven parallelization decisions
- Complexity analysis and risk assessment
- Historical performance tracking
- Feature clustering

**Decision Factors:**
```python
@dataclass
class SchedulerDecision:
    parallelism: int          # 1-5
    feature_ids: list[int]    # Features to run together
    reasoning: str            # Explanation
    confidence: float         # 0-1
    risk_level: str          # "low", "medium", "high"
    estimated_duration: int   # minutes
    warnings: list[str]       # Risk warnings

@dataclass
class FeatureAnalysis:
    feature_id: int
    name: str
    complexity_score: int     # 1-10
    predicted_files: list[str]
    predicted_duration_minutes: int
    risk_factors: list[str]
    can_parallelize_with: list[int]
    should_run_alone: bool
```

**Conflict Detection:**
```python
CONFLICT_CATEGORIES = {
    ("database", "migration"),
    ("database", "schema"),
    ("migration", "schema"),
    ("config", "setup"),
    ("auth", "security"),
}

SAFE_PARALLEL_CATEGORIES = {
    "ui", "frontend", "component", "style", "test", "docs",
}

SOLO_KEYWORDS = [
    "refactor", "restructure", "migrate", "breaking", "schema",
    "database", "init", "setup", "config", "security", "auth",
]
```

### E. Terminal Session (terminal_session.py)

**Responsibilities:**
- Interactive Claude terminal for resource management
- Add/test/configure servers and credentials
- Full system access (sandbox disabled)

**Key Features:**
- Built on Claude Agent SDK
- No sandbox for full system access
- Infrastructure discovery workflows
- Saves resources to JSON (global or project-specific)

---

## 4. MCP SERVER IMPLEMENTATION (mcp_server/feature_mcp.py)

**Server Type:** FastMCP (Anthropic's modern MCP framework)

**Initialization Pattern:**
```python
@asynccontextmanager
async def server_lifespan(server: FastMCP):
    """Initialize database on startup, cleanup on shutdown."""
    global _session_maker, _engine

    # Startup
    PROJECT_DIR.mkdir(parents=True, exist_ok=True)
    _engine, _session_maker = create_database(PROJECT_DIR)
    migrate_json_to_sqlite(PROJECT_DIR, _session_maker)

    yield  # Server runs here

    # Shutdown
    if _engine:
        _engine.dispose()

mcp = FastMCP("features", lifespan=server_lifespan)
```

**Registered Tools:**
```
1. feature_get_stats()
   - Returns: {passing, in_progress, total, percentage}

2. feature_get_next()
   - Returns: Highest-priority pending feature
   - Filters: passes=false, not in_progress

3. feature_get_for_regression(limit: int = 3)
   - Returns: Random passing features for regression testing

4. feature_mark_passing(feature_id: int)
   - Sets: passes=true, clears in_progress

5. feature_skip(feature_id: int)
   - Moves feature to end of queue (reprioritizes)

6. feature_mark_in_progress(feature_id: int)
   - Sets: in_progress=true

7. feature_clear_in_progress(feature_id: int)
   - Clears: in_progress=false

8. feature_create_bulk(features: list[FeatureCreateItem])
   - Creates multiple features atomically
```

**Session Management:**
```python
def get_session() -> Session:
    """Get a new database session (caller must close)."""
    if _session_maker is None:
        raise RuntimeError("Database not initialized")
    return _session_maker()
```

---

## 5. AGENT LOOP STRUCTURE (agent.py)

**Main Session Function:**
```python
async def run_agent_session(
    client: ClaudeSDKClient,
    message: str,
    project_dir: Path,
) -> tuple[str, str]
```

**Session Flow:**
1. Send prompt to Claude via `client.query(message)`
2. Stream response with `client.receive_response()`
3. Handle message types:
   - `AssistantMessage` → text blocks + tool use
   - `UserMessage` → tool results
4. Display output with sanitization
5. Return (status, response_text)

**Response Handling:**
```python
async for msg in client.receive_response():
    msg_type = type(msg).__name__

    if msg_type == "AssistantMessage":
        for block in msg.content:
            if block_type == "TextBlock":
                response_text += block.text
                print(block.text, end="", flush=True)
            elif block_type == "ToolUseBlock":
                print(f"\n[Tool: {block.name}]", flush=True)
                if hasattr(block, "input"):
                    print(f"   Input: {str(block.input)[:200]}", flush=True)

    elif msg_type == "UserMessage":
        for block in msg.content:
            if block_type == "ToolResultBlock":
                is_error = getattr(block, "is_error", False)
                if "blocked" in str(content).lower():
                    print(f"   [BLOCKED] {content}", flush=True)
                elif is_error:
                    print(f"   [Error] {error_str[:500]}", flush=True)
                else:
                    print("   [Done]", flush=True)
```

**Windows Console Encoding:**
```python
if sys.platform == "win32":
    sys.stdout = io.TextIOWrapper(
        sys.stdout.buffer,
        encoding='utf-8',
        errors='replace'
    )
```

---

## 6. UI COMPONENTS & HOOKS

### Hook Pattern: React Query + WebSocket

**WebSocket Hook (useProjectWebSocket):**
```typescript
interface WebSocketState {
  progress: {
    passing: number
    in_progress: number
    total: number
    percentage: number
  }
  agentStatus: AgentStatus
  logs: Array<{ line: string; timestamp: string }>
  isConnected: boolean
}

export function useProjectWebSocket(projectName: string | null) {
  const [state, setState] = useState<WebSocketState>(initialState)
  const wsRef = useRef<WebSocket | null>(null)
  const reconnectAttempts = useRef(0)

  // Features:
  // - Automatic reconnection with exponential backoff
  // - Max 100 log lines cached
  // - Ping every 30 seconds for keep-alive
  // - Message type discrimination
}
```

**Message Types:**
```typescript
type WSMessage =
  | { type: 'progress'; passing: number; in_progress: number; total: number; percentage: number }
  | { type: 'agent_status'; status: AgentStatus }
  | { type: 'log'; line: string; timestamp: string }
  | { type: 'feature_update'; feature_id: number; passes: boolean }
  | { type: 'pong' }
```

**React Query Hooks (useProjects):**
```typescript
export function useProjects() {
  return useQuery({
    queryKey: ['projects'],
    queryFn: api.listProjects,
  })
}

export function useFeatures(projectName: string | null) {
  return useQuery({
    queryKey: ['features', projectName],
    queryFn: () => api.listFeatures(projectName!),
    enabled: !!projectName,
    refetchInterval: 5000,  // Poll every 5 seconds
  })
}

export function useAgentStatus(projectName: string | null) {
  return useQuery({
    queryKey: ['agent-status', projectName],
    queryFn: () => api.getAgentStatus(projectName!),
    enabled: !!projectName,
    refetchInterval: 3000,  // Poll every 3 seconds
  })
}
```

**Mutation Hooks Pattern:**
```typescript
export function useStartAgent(projectName: string) {
  const queryClient = useQueryClient()

  return useMutation({
    mutationFn: (yoloMode: boolean = false) => api.startAgent(projectName, yoloMode),
    onSuccess: () => {
      queryClient.invalidateQueries({ queryKey: ['agent-status', projectName] })
    },
  })
}
```

### Key UI Components:

1. **ProgressDashboard** - Displays percentage, passing/total counts, connection status
2. **KanbanBoard** - Drag-drop feature management (pending/in-progress/done)
3. **FeatureCard** - Individual feature with priority, category, description
4. **AgentControl** - Start/stop/pause/resume buttons with status display
5. **TerminalChat** - Interactive terminal for resource management

---

## 7. PROGRESS TRACKING & SESSION MANAGEMENT

### Progress Module (progress.py)

**Direct Database Access (not API):**
```python
def has_features(project_dir: Path) -> bool:
    """Check if features exist (used before API is ready)."""
    # Checks: features.db has features OR feature_list.json exists

def count_passing_tests(project_dir: Path) -> tuple[int, int, int]:
    """Returns: (passing, in_progress, total)"""
    # Uses sqlite3 directly, handles missing in_progress column

def get_all_passing_features(project_dir: Path) -> list[dict]:
    """Get passing features for webhook notifications."""
    # Returns: [{"id": 1, "category": "...", "name": "..."}, ...]
```

**Progress Webhooks:**
```python
def send_progress_webhook(passing: int, total: int, project_dir: Path) -> None:
    """Sends n8n webhook when progress increases."""
    # Tracks: .progress_cache file
    # Detects: New passing features
    # Payload: {completed_count, newly_passing_tests: [...]}
```

**Cache File:** `.progress_cache` in project directory
```json
{
  "count": 5,
  "passing_ids": [1, 3, 5, 7, 9],
  "timestamp": "2024-01-11T..."
}
```

### Configuration Management (config.py)

**Configuration Structure:**
```python
@dataclass
class GitAgentConfig:
    auto_commit: bool = True
    auto_push: bool = False
    commit_style: str = "conventional"  # conventional|simple|detailed
    push_on: list[str] = ["feature_complete", "session_end"]
    pull_before: list[str] = ["code_review", "feature_start"]

@dataclass
class ParallelConfig:
    enabled: bool = True
    max_agents: int = 3
    lock_timeout_seconds: int = 3600

@dataclass
class SecurityConfig:
    sandbox_enabled: bool = False  # Full system access
    allowed_hosts: list[str] = []  # Empty = all allowed

@dataclass
class PathsConfig:
    credentials_path: str = "~/claude-projects/shared-data/credentials.md"
    global_resources_path: str = "~/.autocoder/global_resources.json"
    session_logs_path: str = "~/claude-projects/session-logs"

@dataclass
class UIConfig:
    theme: str = "neobrutalism"
    show_agent_thoughts: bool = True
    sound_enabled: bool = True
    celebration_enabled: bool = True

@dataclass
class AutocoderConfig:
    version: str = "2.0.0"
    default_model: str = "claude-sonnet-4-20250514"
    git_agent: GitAgentConfig
    parallel: ParallelConfig
    security: SecurityConfig
    paths: PathsConfig
    ui: UIConfig
```

**Storage Location:** `~/.autocoder/config.json`

**Access Pattern:**
```python
def get_config() -> AutocoderConfig:
    """Get global config (singleton)."""

def load_config(path: Optional[Path] = None) -> AutocoderConfig:
    """Load from file or return defaults."""

def save_config(config: Optional[AutocoderConfig] = None) -> None:
    """Save to ~/.autocoder/config.json"""

def update_config(**kwargs) -> AutocoderConfig:
    """Update specific config values."""
```

---

## 8. API ROUTES ORGANIZATION

### FastAPI Application (server/main.py)

**Lifespan Management:**
```python
@asynccontextmanager
async def lifespan(app: FastAPI):
    # Startup: (none - routers handle lazy init)
    yield
    # Shutdown:
    await cleanup_all_managers()
    await cleanup_assistant_sessions()
```

**Middleware Security:**
```python
# CORS: Only localhost
allow_origins = [
    "http://localhost:5173",      # Vite dev
    "http://127.0.0.1:5173",
    "http://localhost:8888",      # Production
    "http://127.0.0.1:8888",
]

# HTTP middleware: Localhost only
@app.middleware("http")
async def require_localhost(request: Request, call_next):
    client_host = request.client.host
    if client_host not in ("127.0.0.1", "::1", "localhost", None):
        raise HTTPException(status_code=403, detail="Localhost access only")
    return await call_next(request)
```

**Router Registration:**
```python
app.include_router(projects_router)      # 15 routes
app.include_router(features_router)      # 12 routes
app.include_router(agent_router)         # 10 routes
app.include_router(spec_creation_router) # 3 routes
app.include_router(filesystem_router)    # 5 routes
app.include_router(assistant_chat_router)# Various
app.include_router(terminal_router)      # Various
app.include_router(git_router)           # Various
app.include_router(scheduler_router)     # Various
app.include_router(system_router)        # Various
```

**WebSocket Endpoint:**
```python
@app.websocket("/ws/projects/{project_name}")
async def websocket_endpoint(websocket: WebSocket, project_name: str):
    await project_websocket(websocket, project_name)
```

**Health & Setup Endpoints:**
```python
@app.get("/api/health") → {"status": "healthy"}
@app.get("/api/setup/status") → SetupStatus
```

**Static File Serving (SPA):**
```python
# Mount /assets for static files
app.mount("/assets", StaticFiles(directory=UI_DIST_DIR / "assets"))

@app.get("/")
async def serve_index():
    return FileResponse(UI_DIST_DIR / "index.html")

@app.get("/{path:path}")
async def serve_spa(path: str):
    # Try file, fall back to index.html for SPA routing
```

---

## 9. WEBSOCKET IMPLEMENTATION

### Connection Manager (server/websocket.py)

**Architecture:**
```python
class ConnectionManager:
    def __init__(self):
        self.active_connections: dict[str, Set[WebSocket]] = {}  # project_name -> connections
        self._lock = asyncio.Lock()

    async def connect(websocket: WebSocket, project_name: str)
    async def disconnect(websocket: WebSocket, project_name: str)
    async def broadcast_to_project(project_name: str, message: dict)
    def get_connection_count(project_name: str) -> int
```

**Main Handler:**
```python
async def project_websocket(websocket: WebSocket, project_name: str):
    # Validate project name (path traversal prevention)
    if not validate_project_name(project_name):
        await websocket.close(code=4000, reason="Invalid project name")
        return

    # Get project manager
    agent_manager = get_manager(project_name, project_dir, ROOT_DIR)

    # Register callbacks
    agent_manager.add_output_callback(on_output)
    agent_manager.add_status_callback(on_status_change)

    # Start polling task
    poll_task = asyncio.create_task(poll_progress(...))

    # Send initial state
    await websocket.send_json({
        "type": "agent_status",
        "status": agent_manager.status,
    })

    # Handle incoming messages (ping, commands)
    while True:
        try:
            data = await websocket.receive_text()
            message = json.loads(data)
            if message.get("type") == "ping":
                await websocket.send_json({"type": "pong"})
        except WebSocketDisconnect:
            break
```

**Progress Polling:**
```python
async def poll_progress(websocket: WebSocket, project_name: str, project_dir: Path):
    """Poll database every 2 seconds for progress changes."""
    last_passing = -1
    last_in_progress = -1
    last_total = -1

    while True:
        passing, in_progress, total = count_passing_tests(project_dir)

        # Only send if changed
        if passing != last_passing or in_progress != last_in_progress or total != last_total:
            percentage = (passing / total * 100) if total > 0 else 0
            await websocket.send_json({
                "type": "progress",
                "passing": passing,
                "in_progress": in_progress,
                "total": total,
                "percentage": round(percentage, 1),
            })

        await asyncio.sleep(2)
```

---

## 10. PROCESS MANAGEMENT & CLEANUP PATTERNS

### Lifecycle Management

**Startup (implicit in routers):**
- Database created on first access
- Process managers created on first project access
- Registry loaded on first project access

**Shutdown:**
```python
async def cleanup_all_managers() -> None:
    """Stop all running agents (called on server shutdown)."""
    with _managers_lock:
        managers = list(_managers.values())

    for manager in managers:
        if manager.status != "stopped":
            await manager.stop()

    with _managers_lock:
        _managers.clear()

async def cleanup_assistant_sessions() -> None:
    """Stop all assistant chat sessions."""
    # Similar pattern for chat sessions
```

**Orphan Process Cleanup:**
```python
def find_orphan_processes() -> list[dict]:
    """Find Autocoder processes with missing parent."""
    patterns = [
        "autonomous_agent_demo.py",
        "feature_mcp",
        "@playwright/mcp",
        "mcp_server",
    ]

    for proc in psutil.process_iter(['pid', 'cmdline', 'ppid']):
        ppid = info.get('ppid', 0)
        is_orphan = ppid in (0, 1) or not psutil.pid_exists(ppid)

        if is_orphan and any(p in cmdline for p in patterns):
            orphans.append(...)

def kill_orphan_processes() -> dict:
    """Kill orphaned Autocoder processes."""
    # Returns: {found, killed, failed, details}
```

---

## 11. CONFIGURATION MANAGEMENT PATTERNS

### Lazy Initialization Pattern

Used throughout to avoid circular dependencies:

```python
# In routers
_imports_initialized = False
_check_spec_exists = None

def _init_imports():
    global _imports_initialized, _check_spec_exists
    if _imports_initialized:
        return

    import sys
    root = Path(__file__).parent.parent.parent
    if str(root) not in sys.path:
        sys.path.insert(0, str(root))

    from progress import count_passing_tests
    from prompts import get_project_prompts_dir

    _check_spec_exists = check_spec_exists
    _imports_initialized = True
```

### Registry Functions Pattern

```python
def _get_registry_functions():
    """Get registry functions with lazy import."""
    import sys
    root = Path(__file__).parent.parent.parent
    if str(root) not in sys.path:
        sys.path.insert(0, str(root))

    from registry import (
        get_project_path,
        list_registered_projects,
        register_project,
    )
    return register_project, list_registered_projects, get_project_path

# In endpoint
register_project, _, get_project_path = _get_registry_functions()
```

---

## 12. KEY ARCHITECTURAL PRINCIPLES

### 1. **Cross-Platform Path Handling**
- Use POSIX-style paths (forward slashes) internally
- Convert using `.as_posix()` for SQLite URLs
- `Path` objects for OS-independent operations

### 2. **Lazy Imports**
- Avoid circular dependencies between routers and services
- Initialize on first access
- Use lazy import functions in routers

### 3. **Singleton Managers with Thread Safety**
- Global registry with `threading.Lock()`
- Reuse manager instances across requests
- Proper cleanup on shutdown

### 4. **Multiple Callback Support**
- Process manager supports multiple WebSocket clients
- Thread-safe callback registration/deregistration
- Async callback execution in event loop

### 5. **Security**
- Localhost-only HTTP middleware
- Bash output sanitization (redact API keys, tokens)
- Project name validation (regex, path traversal prevention)
- Sandbox disabled for full system access

### 6. **Real-Time Updates**
- WebSocket for progress streaming
- Polling for fallback/complex updates
- Redis-like message broadcasting
- Keep-alive ping/pong mechanism

### 7. **Database Access Patterns**
- Direct SQLite access in progress module (no API dependency)
- FastAPI dependency injection for sessions
- Automatic session cleanup via context managers
- Migration support for schema changes

### 8. **Error Handling**
- Try/catch in callbacks to prevent cascade failures
- Graceful degradation (offline mode)
- Detailed error messages in responses
- Logging for debugging

---

## RECOMMENDATIONS FOR ENHANCEMENTS

To maintain architectural consistency when adding features:

1. **New Services**: Follow the singleton+lock pattern in `__init__.py`
2. **New Routers**: Use lazy imports to avoid circular dependencies
3. **New Database Models**: Extend `api/database.py` with new classes
4. **WebSocket Messages**: Add to message type discrimination in handler
5. **UI Hooks**: Use React Query with appropriate refetch intervals
6. **Process Management**: Extend `AgentProcessManager` with new methods
7. **Configuration**: Add sections to `config.py` dataclasses
8. **Cleanup**: Ensure shutdown handlers in `lifespan()` or router cleanup

---

**Key File Locations Summary:**

| Component | Path |
|-----------|------|
| Database Models | `/c/Users/MikeSwanson/claude-projects/Autocode-remix/Autocoder-2.0/api/database.py` |
| Process Manager | `/c/Users/MikeSwanson/claude-projects/Autocode-remix/Autocoder-2.0/server/services/process_manager.py` |
| WebSocket Handler | `/c/Users/MikeSwanson/claude-projects/Autocode-remix/Autocoder-2.0/server/websocket.py` |
| FastAPI App | `/c/Users/MikeSwanson/claude-projects/Autocode-remix/Autocoder-2.0/server/main.py` |
| Schemas/Models | `/c/Users/MikeSwanson/claude-projects/Autocode-remix/Autocoder-2.0/server/schemas.py` |
| Feature Router | `/c/Users/MikeSwanson/claude-projects/Autocode-remix/Autocoder-2.0/server/routers/features.py` |
| Agent Router | `/c/Users/MikeSwanson/claude-projects/Autocode-remix/Autocoder-2.0/server/routers/agent.py` |
| Projects Router | `/c/Users/MikeSwanson/claude-projects/Autocode-remix/Autocoder-2.0/server/routers/projects.py` |
| React Hooks | `/c/Users/MikeSwanson/claude-projects/Autocode-remix/Autocoder-2.0/ui/src/hooks/` |
| UI Components | `/c/Users/MikeSwanson/claude-projects/Autocode-remix/Autocoder-2.0/ui/src/components/` |

agentId: a470bd6 (for resuming to continue this agent's work if needed)

Prompt:
Explore the Autocoder 2.0 codebase to understand:

1. Current project structure and key files
2. Database schema in api/database.py
3. Existing services architecture in server/services/
4. MCP server implementation
5. Agent loop structure in agent.py
6. UI components and hooks structure
7. Progress tracking and session management

Focus on understanding the existing patterns so I can implement enhancements that follow the same architectural style. Look for:
- How services are structured and initialized
- Database model patterns
- API route organization
- WebSocket implementation for real-time updates
- Process management and cleanup patterns
- Configuration management

This will help me design enhancements that integrate cleanly with the existing system.


Now I understand the architecture thoroughly! This is a sophisticated system with clean patterns. Let me propose an implementation plan for your priority enhancements.


Entered plan mode. You should now focus on exploring the codebase and designing an implementation approach.

In plan mode, you should:
1. Thoroughly explore the codebase to understand existing patterns
2. Identify similar features and architectural approaches
3. Consider multiple approaches and their trade-offs
4. Use AskUserQuestion if you need to clarify the approach
5. Design a concrete implementation strategy
6. When ready, use ExitPlanMode to present your plan for approval

Remember: DO NOT write or edit any files yet. This is a read-only exploration and planning phase.


Now I'll design a comprehensive implementation plan for enhancing Autocoder 2.0. Based on my exploration, I understand the system architecture and can plan enhancements that follow the existing patterns.

Let me start by exploring the specific areas that will need enhancement for the priority features:


Explore token tracking opportunities

Prompt:
Explore the Autocoder 2.0 codebase to understand how to implement token and cost tracking:

1. **Current Claude SDK usage**: Find where Claude API calls are made (`client.py`, `agent.py`) and what response metadata is available for token counting

2. **Database patterns**: Look at `api/database.py` to understand the SQLAlchemy model patterns for adding new tables (Usage, SessionData, etc.)

3. **WebSocket real-time updates**: Examine `server/websocket.py` to understand how to add new message types for real-time cost updates

4. **Session management**: Look at how sessions are currently tracked and where we could inject token counting

5. **UI integration points**: Find where cost/usage data could be displayed in the React components (header, dashboard)

Focus on finding the specific integration points and existing patterns I can extend for token tracking without breaking the current functionality.


Perfect! Now I have comprehensive understanding of the codebase. Let me create a detailed analysis document for the user.

## Comprehensive Autocoder 2.0 Token & Cost Tracking Analysis

Based on my exploration of the Autocoder 2.0 codebase, here's a detailed breakdown of integration points and patterns for implementing token and cost tracking:

---

### 1. CURRENT CLAUDE SDK USAGE & RESPONSE METADATA

**Files:** `client.py`, `agent.py`, `server/services/assistant_chat_session.py`

**Current Implementation:**
- Claude SDK client initialized via `ClaudeSDKClient` with `ClaudeAgentOptions` (lines 183-201 in `client.py`)
- API calls made through `client.query(message)` and `client.receive_response()` (agent.py:56, 60)
- Streaming responses via async generator: `async for msg in client.receive_response()`

**Token/Cost Data Available:**
According to Claude SDK documentation, the `receive_response()` stream yields message objects with:
- `AssistantMessage` objects containing content blocks
- Each response should have metadata with token usage (input/output tokens)
- **Key integration point**: Messages are streamed, so token metadata needs to be extracted per-response cycle

**Current Response Handling Pattern** (agent.py:60-98):
```python
async for msg in client.receive_response():
    msg_type = type(msg).__name__
    if msg_type == "AssistantMessage" and hasattr(msg, "content"):
        for block in msg.content:
            # Process TextBlock and ToolUseBlock
    elif msg_type == "UserMessage" and hasattr(msg, "content"):
        # Process ToolResultBlock
```

**What's Missing:**
- No extraction of `usage` metadata from response messages
- No tracking of `stop_reason` (e.g., "end_turn", "tool_use", "max_tokens")
- SDK hook for final response metrics not being used

**Integration Points for Token Tracking:**
1. After `client.receive_response()` completes → extract final usage stats
2. Hook into message objects' `.usage` attribute (if available on final message)
3. Alternative: Use `PostToolUse` or custom hooks (client.py:192-196 shows hook pattern)

---

### 2. DATABASE PATTERNS FOR NEW TABLES

**File:** `api/database.py`

**Current SQLAlchemy Model Pattern:**
```python
class Feature(Base):
    __tablename__ = "features"

    id = Column(Integer, primary_key=True, index=True)
    priority = Column(Integer, nullable=False, default=999, index=True)
    category = Column(String(100), nullable=False)
    # ... more columns

    def to_dict(self) -> dict:
        return {...}
```

**Database Setup Functions:**
- `create_database(project_dir)` → returns `(engine, SessionLocal)` tuple
- `get_database_url(project_dir)` → constructs SQLite URL with POSIX paths
- `_migrate_add_in_progress_column(engine)` → shows migration pattern using `ALTER TABLE`
- Global session maker pattern: `set_session_maker()` and `get_db()` dependency

**Recommended Tables for Token Tracking:**

**Table 1: `usage_sessions` (per agent run)**
```python
class UsageSession(Base):
    __tablename__ = "usage_sessions"

    id = Column(Integer, primary_key=True)
    session_id = Column(String(100), unique=True, index=True)  # UUID
    started_at = Column(DateTime, default=datetime.utcnow, index=True)
    ended_at = Column(DateTime, nullable=True)

    # Cumulative totals
    total_input_tokens = Column(Integer, default=0)
    total_output_tokens = Column(Integer, default=0)
    total_calls = Column(Integer, default=0)

    model = Column(String(50))  # e.g., "claude-opus-4-5"
    status = Column(String(20))  # "active", "completed", "failed"
```

**Table 2: `api_calls` (per Claude API call)**
```python
class APICall(Base):
    __tablename__ = "api_calls"

    id = Column(Integer, primary_key=True)
    session_id = Column(String(100), ForeignKey("usage_sessions.session_id"), index=True)
    timestamp = Column(DateTime, default=datetime.utcnow, index=True)

    # Input
    call_type = Column(String(50))  # "query", "message", "tool_result"
    prompt_length = Column(Integer)

    # Output
    input_tokens = Column(Integer)
    output_tokens = Column(Integer)
    stop_reason = Column(String(50))  # "end_turn", "tool_use", "max_tokens"
    response_time_ms = Column(Integer)

    # Cost calculation
    cost_cents = Column(Integer)  # Store as cents to avoid float precision issues
```

**Table 3: `usage_summary` (cached daily/session stats)**
```python
class UsageSummary(Base):
    __tablename__ = "usage_summary"

    id = Column(Integer, primary_key=True)
    session_id = Column(String(100), unique=True, index=True)

    total_cost_cents = Column(Integer, default=0)
    total_input_tokens = Column(Integer, default=0)
    total_output_tokens = Column(Integer, default=0)
    total_calls = Column(Integer, default=0)

    # For display
    cost_display = Column(String(20))  # "$12.34"
    tokens_display = Column(String(50))  # "45k input / 12k output"
```

**Migration Pattern:**
```python
def _migrate_add_usage_tables(engine) -> None:
    """Add usage tracking tables to existing databases."""
    from sqlalchemy import text
    with engine.connect() as conn:
        # Check if table exists
        result = conn.execute(text("SELECT name FROM sqlite_master WHERE type='table' AND name='api_calls'"))
        if not result.fetchone():
            # Table doesn't exist, create via Base.metadata.create_all()
            UsageSession.__table__.create(engine, checkfirst=True)
            APICall.__table__.create(engine, checkfirst=True)
            conn.commit()
```

---

### 3. WEBSOCKET REAL-TIME UPDATES

**File:** `server/websocket.py`

**Current WebSocket Message Types:**
```python
{
    "type": "progress",
    "passing": int,
    "in_progress": int,
    "total": int,
    "percentage": float
}

{
    "type": "agent_status",
    "status": str
}

{
    "type": "log",
    "line": str,
    "timestamp": datetime
}
```

**How WebSocket Architecture Works:**
- `ConnectionManager` class manages per-project WebSocket connections (lines 50-98)
- `broadcast_to_project(project_name, message)` sends to all connections (lines 75-93)
- `poll_progress()` polls database every 2 seconds for progress updates (lines 112-143)
- Callbacks system for agent output: `agent_manager.add_output_callback()` (lines 195-196)

**Integration Points for Token Updates:**

**1. Add new WebSocket message type:**
```python
{
    "type": "usage_update",
    "session_id": str,
    "total_input_tokens": int,
    "total_output_tokens": int,
    "total_calls": int,
    "estimated_cost": "$X.XX",
    "timestamp": datetime
}
```

**2. Polling approach (similar to progress):**
```python
async def poll_usage(websocket: WebSocket, project_name: str, session_id: str):
    """Poll usage database for cost/token updates."""
    last_total_calls = -1

    while True:
        try:
            usage = get_usage_summary(session_id)  # Query API_CALLS table

            if usage['total_calls'] != last_total_calls:
                last_total_calls = usage['total_calls']
                await websocket.send_json({
                    "type": "usage_update",
                    "total_input_tokens": usage['input_tokens'],
                    "total_output_tokens": usage['output_tokens'],
                    "total_calls": usage['total_calls'],
                    "estimated_cost": format_cost(usage['cost_cents']),
                    "timestamp": datetime.now().isoformat(),
                })

            await asyncio.sleep(1)  # Poll every second
        except asyncio.CancelledError:
            raise
        except Exception as e:
            logger.warning(f"Usage polling error: {e}")
            break
```

**3. Callback approach (real-time on each API call):**
```python
# In agent process manager (process_manager.py)
def _notify_usage_update(self, usage_data: dict) -> None:
    """Notify all registered callbacks of usage update."""
    with self._callbacks_lock:
        callbacks = list(self._usage_callbacks)

    for callback in callbacks:
        try:
            loop = asyncio.get_running_loop()
            loop.create_task(self._safe_callback(callback, usage_data))
        except RuntimeError:
            pass

# In WebSocket handler
async def on_usage_update(data: dict):
    await websocket.send_json({
        "type": "usage_update",
        **data,
        "timestamp": datetime.now().isoformat(),
    })

agent_manager.add_usage_callback(on_usage_update)
```

---

### 4. SESSION MANAGEMENT & TOKEN INJECTION POINTS

**Files:** `agent.py`, `server/services/assistant_chat_session.py`, `server/services/process_manager.py`

**Current Session Architecture:**

1. **Agent Sessions** (autonomous_agent_demo.py):
   - One agent process per project
   - Continuous loop with multiple iterations (agent.py:164-209)
   - Each iteration creates fresh client context (agent.py:177)

2. **Assistant Chat Sessions** (assistant_chat_session.py):
   - Per-project conversational assistant sessions
   - Uses `AssistantChatSession` class with session registry (lines 99-373)
   - Persists to `assistant.db` (ConversationMessage table)

**Where to Inject Token Counting:**

**Option A: Agent Loop Level** (agent.py:34-105)
```python
# Before: await client.query(message)
session_id = generate_session_id()
start_time = time.time()

await client.query(message)

# After: async for msg in client.receive_response()
usage_data = extract_usage_from_final_message()  # NEW
track_api_call(
    session_id=session_id,
    call_type="query",
    input_tokens=usage_data['input_tokens'],
    output_tokens=usage_data['output_tokens'],
    duration_ms=int((time.time() - start_time) * 1000),
    cost_cents=calculate_cost(usage_data)
)
```

**Option B: Client Hook Level** (client.py:192-196)
```python
# Add PostToolUse hook to capture token data after each tool execution
hooks={
    "PostToolUse": [
        HookMatcher(matcher="*", hooks=[token_tracking_hook]),
    ]
}

def token_tracking_hook(hook_input) -> None:
    """Hook called after tool execution to capture token usage."""
    # Extract usage metadata from hook_input
    if hasattr(hook_input, 'usage'):
        track_api_call(hook_input.usage)
```

**Option C: Assistant Chat Session Level** (assistant_chat_session.py:255-294)
```python
# In _query_claude() method, after response stream completes
async def _query_claude(self, message: str):
    start_time = time.time()

    await self.client.query(message)
    full_response = ""

    async for msg in self.client.receive_response():
        # ... existing code ...

    # NEW: Extract and track usage
    if hasattr(msg, 'usage'):
        track_conversation_usage(
            conversation_id=self.conversation_id,
            input_tokens=msg.usage.input_tokens,
            output_tokens=msg.usage.output_tokens,
            duration_ms=int((time.time() - start_time) * 1000),
        )
```

**Session Tracking Context:**

The assistant_database pattern shows good session persistence:
```python
class Conversation(Base):
    __tablename__ = "conversations"
    id = Column(Integer, primary_key=True)
    project_name = Column(String(100), index=True)
    created_at = Column(DateTime)
    # Add usage tracking
    # total_tokens_used = Column(Integer, default=0)
    # total_cost_cents = Column(Integer, default=0)
```

---

### 5. UI INTEGRATION POINTS

**Files:** `ui/src/App.tsx`, `ui/src/components/ProgressDashboard.tsx`, `ui/src/hooks/useWebSocket.ts`

**Current UI Message Flow:**

1. **WebSocket Types** (useWebSocket.ts:8-18):
```typescript
interface WebSocketState {
  progress: { passing, in_progress, total, percentage }
  agentStatus: AgentStatus
  logs: Array<{ line, timestamp }>
  isConnected: boolean
}
```

2. **Message Handler** (useWebSocket.ts:51-92):
```typescript
switch (message.type) {
    case 'progress': setState(...)
    case 'agent_status': setState(...)
    case 'log': setState(...)
    case 'feature_update': // triggers refetch
    case 'pong': // heartbeat
}
```

**Integration Points for Cost Display:**

**1. Extend WebSocket State** (useWebSocket.ts):
```typescript
interface WebSocketState {
  progress: { ... }
  agentStatus: AgentStatus
  logs: Array<{ line, timestamp }>
  isConnected: boolean

  // NEW: Usage tracking
  usage: {
    totalInputTokens: number
    totalOutputTokens: number
    totalCalls: number
    estimatedCost: string  // "$12.34"
    lastUpdated: string
  }
}
```

**2. Add message handler** (useWebSocket.ts:51-92):
```typescript
case 'usage_update':
    setState(prev => ({
        ...prev,
        usage: {
            totalInputTokens: message.total_input_tokens,
            totalOutputTokens: message.total_output_tokens,
            totalCalls: message.total_calls,
            estimatedCost: message.estimated_cost,
            lastUpdated: message.timestamp,
        }
    }))
    break
```

**3. Display in Header or Dashboard:**

**Option A: Cost Badge in Header** (App.tsx:160-239)
```typescript
// Add next to AgentControl component
{selectedProject && (
    <div className="flex items-center gap-2 px-3 py-2 bg-black/20 rounded border border-white/20">
        <DollarSign size={16} />
        <span className="font-mono text-sm">{wsState.usage.estimatedCost}</span>
        <span className="text-xs text-white/60">{wsState.usage.totalCalls} calls</span>
    </div>
)}
```

**Option B: Extended Progress Dashboard** (ProgressDashboard.tsx)
```typescript
// Below percentage display, add usage section
<div className="border-t border-[var(--color-neo-border)] pt-4 mt-4">
    <h3 className="font-display text-sm uppercase mb-3">Usage</h3>
    <div className="grid grid-cols-3 gap-4 text-center">
        <div>
            <span className="font-mono text-lg">{usage.totalInputTokens.toLocaleString()}</span>
            <span className="block text-xs text-[var(--color-neo-text-secondary)]">Input Tokens</span>
        </div>
        <div>
            <span className="font-mono text-lg">{usage.totalOutputTokens.toLocaleString()}</span>
            <span className="block text-xs text-[var(--color-neo-text-secondary)]">Output Tokens</span>
        </div>
        <div>
            <span className="font-mono text-lg text-[var(--color-neo-done)]">{usage.estimatedCost}</span>
            <span className="block text-xs text-[var(--color-neo-text-secondary)]">Est. Cost</span>
        </div>
    </div>
</div>
```

**Option C: Separate Usage Panel** (new component)
```typescript
// UsagePanel.tsx
export function UsagePanel({ usage }: { usage: UsageData }) {
    return (
        <div className="neo-card p-4">
            <h2 className="font-display font-bold mb-4">API Usage</h2>
            <div className="space-y-3">
                <div className="flex justify-between">
                    <span>Tokens</span>
                    <span className="font-mono">{total} tokens</span>
                </div>
                <div className="flex justify-between">
                    <span>API Calls</span>
                    <span className="font-mono">{usage.totalCalls}</span>
                </div>
                <div className="flex justify-between border-t pt-3">
                    <span className="font-bold">Estimated Cost</span>
                    <span className="font-mono font-bold text-[var(--color-neo-done)]">
                        {usage.estimatedCost}
                    </span>
                </div>
            </div>
        </div>
    )
}
```

**API Endpoints for Cost Data:**

Could add REST endpoint for historical data:
```python
# server/routers/usage.py (NEW)
@router.get("/api/projects/{project_name}/usage/summary")
async def get_usage_summary(project_name: str):
    """Get usage summary for a project."""
    return {
        "total_input_tokens": int,
        "total_output_tokens": int,
        "total_calls": int,
        "estimated_cost": str,
        "calls_by_day": [...],
    }

@router.get("/api/projects/{project_name}/usage/timeline")
async def get_usage_timeline(project_name: str, days: int = 7):
    """Get hourly usage timeline."""
    return [
        {
            "timestamp": datetime,
            "input_tokens": int,
            "output_tokens": int,
            "calls": int,
            "cost_cents": int,
        }
    ]
```

---

### 6. COST CALCULATION UTILITIES

**Pricing Constants** (should be in `config.py` or new `pricing.py`):
```python
# Current Claude API pricing (Opus 4.5)
PRICING = {
    "claude-opus-4-5": {
        "input": 0.00015,  # $0.15 per 1M input tokens
        "output": 0.0006,   # $0.60 per 1M output tokens
    },
    "claude-opus-4-5-sonnet": {
        "input": 0.000003,
        "output": 0.000015,
    },
    "claude-3-5-haiku": {
        "input": 0.000008,
        "output": 0.00004,
    },
}

def calculate_cost(input_tokens: int, output_tokens: int, model: str) -> int:
    """Calculate cost in cents."""
    rates = PRICING.get(model, PRICING["claude-opus-4-5"])
    cost_dollars = (input_tokens * rates["input"]) + (output_tokens * rates["output"])
    return int(round(cost_dollars * 100))  # Convert to cents

def format_cost(cents: int) -> str:
    """Format cost as currency string."""
    return f"${cents / 100:.2f}"
```

---

### 7. IMPLEMENTATION SUMMARY & MINIMAL BREAKING CHANGES

**Phase 1: Database Setup (No Breaking Changes)**
1. Add migration in `api/migration.py` or `api/database.py`
2. Create `Usage`, `APICall`, `UsageSummary` models
3. Initialize tables on server startup

**Phase 2: Token Extraction (Minimal Changes)**
1. Modify `agent.py` to capture usage after `receive_response()`
2. Modify `assistant_chat_session.py` to track tokens per message
3. Store in database via new functions

**Phase 3: WebSocket Broadcasting (Isolated)**
1. Add `usage_callbacks` to `AgentProcessManager` (follows existing pattern)
2. Add new message type to `websocket.py` poll function
3. Keep existing message types unchanged

**Phase 4: UI Enhancement (Non-Breaking)**
1. Extend `WebSocketState` type in `useWebSocket.ts`
2. Add new case in message handler switch statement
3. Add usage display components (new files, no changes to existing components)

**Phase 5: REST API (Optional)**
1. Create new router `server/routers/usage.py`
2. Query usage tables for historical data
3. Independent from existing API

---

### 8. KEY FILES TO MODIFY (In Priority Order)

| File | Change Type | Purpose |
|------|-------------|---------|
| `api/database.py` | Add models | `Usage`, `APICall`, `UsageSummary` tables |
| `agent.py` | Insert tracking | Extract tokens after each response |
| `server/services/assistant_chat_session.py` | Insert tracking | Extract tokens in `_query_claude()` |
| `server/websocket.py` | Add polling | `poll_usage()` function, new message type |
| `ui/src/hooks/useWebSocket.ts` | Extend state | Add `usage` field to `WebSocketState` |
| `ui/src/components/ProgressDashboard.tsx` | Add display | Usage stats section (or new component) |
| `config.py` | Add pricing | Pricing constants and calculator |

---

### 9. EXISTING PATTERNS YOU CAN REUSE

✅ **Message Type System**: WebSocket already uses `type` field for routing (useWebSocket.ts:55-92)

✅ **Polling Pattern**: `poll_progress()` model can be copied for `poll_usage()` (websocket.py:112-143)

✅ **Callback System**: `AgentProcessManager` already has output/status callbacks (process_manager.py:118-136)

✅ **Database Migration**: `_migrate_add_in_progress_column()` shows safe migration pattern (database.py:61-73)

✅ **Session Management**: Both agent and assistant already track sessions (registry.py, assistant_database.py)

✅ **TypeScript Message Types**: `WSMessage` union type can be extended (useWebSocket.ts:6)

✅ **SQLAlchemy ORM**: Established models with `to_dict()` serialization (database.py:33-44, assistant_database.py)

---

This exploration provides everything you need to extend Autocoder 2.0 with token and cost tracking while maintaining backward compatibility and leveraging existing architectural patterns. The integration points are well-defined and isolated, making implementation straightforward without breaking current functionality.

agentId: ae9ce56 (for resuming to continue this agent's work if needed)

Prompt:
Explore the Autocoder 2.0 codebase to understand how to implement token and cost tracking:

1. **Current Claude SDK usage**: Find where Claude API calls are made (`client.py`, `agent.py`) and what response metadata is available for token counting

2. **Database patterns**: Look at `api/database.py` to understand the SQLAlchemy model patterns for adding new tables (Usage, SessionData, etc.)

3. **WebSocket real-time updates**: Examine `server/websocket.py` to understand how to add new message types for real-time cost updates

4. **Session management**: Look at how sessions are currently tracked and where we could inject token counting

5. **UI integration points**: Find where cost/usage data could be displayed in the React components (header, dashboard)

Focus on finding the specific integration points and existing patterns I can extend for token tracking without breaking the current functionality.


Explore session recovery opportunities

Prompt:
Explore the Autocoder 2.0 codebase to understand how to implement session recovery/checkpointing:

1. **Current session state**: Look at `agent.py`, `progress.py`, and `server/services/process_manager.py` to understand what state exists during an agent session and where it's stored

2. **Process lifecycle**: Examine how agent sessions are started, stopped, and tracked - specifically the lock file mechanism and PID tracking in `process_manager.py`

3. **Feature state management**: Look at how feature progress is currently persisted (database writes, in_progress flag) and at what points we could create checkpoints

4. **MCP server state**: Examine `mcp_server/feature_mcp.py` to see how the MCP tools track and resume work on features

5. **Existing persistence patterns**: Find where the system already saves state to disk (database, lock files, config) and what patterns we can follow

Focus on identifying what state would need to be saved for recovery and the best checkpoint triggers (after tool use, after feature completion, etc.).


Perfect! Now I have a comprehensive understanding of the codebase. Let me create a detailed analysis report of the session recovery/checkpointing architecture.

## Session Recovery/Checkpointing Analysis for Autocoder 2.0

Based on my thorough exploration of the Autocoder 2.0 codebase, here's a comprehensive analysis of the current session state management and recommendations for implementing session recovery/checkpointing:

---

### 1. CURRENT SESSION STATE ARCHITECTURE

#### A. Agent Session Lifecycle (`agent.py`)
The agent operates in a fresh context for each iteration:
- **No persistent message history**: Each iteration creates a new `ClaudeSDKClient` from scratch
- **Feature-driven iteration**: Uses `is_first_run` flag to determine if initializer or coding agent should run
- **Detection mechanism**: `has_features(project_dir)` checks if `features.db` has records to distinguish between fresh starts and continuations
- **Auto-continue pattern**: After each session completes, auto-continues after 3-second delay if status == "continue"

**Key limitation**: While feature state persists in the database, the Claude agent context is completely discarded between sessions. There's no memory of what work was done, what approaches were tried, or what blockers were encountered.

#### B. Process Lifecycle Management (`process_manager.py`)
**Lock file mechanism** (`.agent.lock`):
- Stores PID of running agent process
- Prevents multiple agents from running on same project
- Checked on startup; stale locks cleaned up
- Lock removed on process termination

**State tracking**:
```python
_status: Literal["stopped", "running", "paused", "crashed"]
started_at: datetime
yolo_mode: bool
pid: int
```

**Process tree killing**: Handles cleanup of agent + MCP servers + child processes on stop

**Health checking**: Can detect crashed processes and update status accordingly

#### C. Feature State Persistence (`api/database.py` + `mcp_server/feature_mcp.py`)

**Feature database schema** (SQLite):
```python
id: Integer (primary key)
priority: Integer (index)
category: String
name: String
description: Text
steps: JSON (array of strings)
passes: Boolean (index)  # Marks completion
in_progress: Boolean (index)  # Marks active work
```

**MCP tools for state mutation**:
- `feature_mark_passing(feature_id)` - Mark complete
- `feature_mark_in_progress(feature_id)` - Lock feature for work
- `feature_clear_in_progress(feature_id)` - Unlock on failure
- `feature_skip(feature_id)` - Move to end of queue
- `feature_get_next()` - Get highest-priority pending feature
- `feature_get_stats()` - Get passing/in_progress/total counts

**Progress caching** (`progress.py`):
- Stores passing feature IDs in `.progress_cache` JSON file
- Used to detect new passing tests and send webhooks
- Backup: Database queries directly count features with `passes=1`

#### D. File Locking for Parallel Work (`file_lock.py`)

**In-memory lock manager**:
- Async-safe, per-file locks with expiration
- Supports timeouts (default 1 hour)
- Used to coordinate parallel agents
- Lock info: `file_path`, `feature_id`, `agent_id`, `locked_at`, `expires_at`
- Can be force-released for recovery

**Limitation**: Locks are in-memory only, lost on process restart

#### E. Migration/Backward Compatibility (`api/migration.py`)

**JSON to SQLite migration**:
- Detects legacy `feature_list.json`
- Imports to database if DB is empty
- Renames JSON to backup with timestamp
- Enables smooth upgrades

**Export functionality**: Can export features back to JSON for debugging

---

### 2. WHAT STATE EXISTS DURING AN AGENT SESSION

#### Session State (Ephemeral - Lost on Exit)
```
Claude SDK Client Context:
├── Message history (lost after session)
├── Tool execution results (lost after session)
├── Current working context (lost after session)
└── Intermediate file state (persisted via Write/Edit tools)
```

#### Persistent State (Disk-based)
```
Project Directory/
├── features.db (SQLite)
│   ├── Feature table with passes/in_progress flags
│   ├── Auto-incremented IDs for each feature
│   └── Priority ordering
├── .agent.lock (PID file)
├── .progress_cache (JSON - passing feature IDs)
├── Generated code files (created by agent)
├── .claude_settings.json (security config per session)
└── [project files created/modified by agent]
```

---

### 3. KEY CHECKPOINT TRIGGERS & RECOVERY POINTS

#### Natural Checkpoint Opportunities
1. **After tool use completes** (Write/Edit/Bash)
   - Tool result is received
   - State before tool can be saved
   - Recovery can resume from tool result

2. **After feature marked passing**
   - `feature_mark_passing(feature_id)` called
   - Database transaction committed
   - Next session picks up at `feature_get_next()`

3. **After feature marked in_progress**
   - `feature_mark_in_progress(feature_id)` called
   - Prevents duplicate work if session crashes
   - Recovery can detect stuck features via stale in_progress flags

4. **On session start**
   - Check for interrupted features (in_progress=1 but not by current session)
   - Could resume work or reset stale flags

5. **On session end**
   - Summary of work completed
   - Reset any abandoned in_progress features
   - Create checkpoint for next iteration

---

### 4. RECOMMENDATIONS FOR RECOVERY/CHECKPOINTING SYSTEM

#### A. Conversation History Checkpoint System

**Create a session journal** (`project_dir/.session_journal/`):
```
.session_journal/
├── session_001.json          # Structured log of first session
├── session_002.json          # Structured log of second session
└── current_session.json      # Active session being written to

Schema per session:
{
  "session_id": "uuid",
  "started_at": "ISO timestamp",
  "ended_at": "ISO timestamp",
  "iteration": 1,
  "is_initializer": false,
  "feature_id": 42,
  "initial_stats": { "passing": 10, "in_progress": 2, "total": 50 },
  "final_stats": { "passing": 12, "in_progress": 0, "total": 50 },
  "messages": [
    {
      "role": "user",
      "content": "...",
      "timestamp": "ISO"
    },
    {
      "role": "assistant",
      "content": "...",
      "tools_used": ["feature_mark_passing"],
      "timestamp": "ISO"
    }
  ],
  "blockers": ["Dependency on feature #5", "..."],
  "decisions": ["Skipped feature #10 due to unclear requirements"],
  "errors": ["Tool X failed with error Y"],
  "status": "success|incomplete|error",
  "checkpoints": [
    {
      "timestamp": "ISO",
      "trigger": "after_tool_use|after_feature_complete",
      "tool_name": "Write",
      "state_summary": "..."
    }
  ]
}
```

**Benefits**:
- Can construct conversation summary for next session
- Helps agent learn from previous decisions
- Enables "session replay" for debugging
- Tracks what didn't work to avoid retry

#### B. Feature Work State Tracking

**Extend Feature model** with recovery metadata:
```python
# Add columns to features table:
last_worked_on: DateTime        # When feature was last attempted
work_session_id: String         # Which session last worked on it
attempt_count: Integer          # How many times attempted
last_error: Text                # Last error encountered
blocked_by_features: JSON       # List of feature IDs blocking this
recovery_notes: Text            # Notes from agent about state
```

**Benefits**:
- Detect truly stuck features
- Avoid infinite retry loops
- Track which sessions touched which features
- Support intelligent recovery decisions

#### C. Tool Execution Checkpoint

**Track tool results** in session journal:
```python
{
  "tool_execution": {
    "sequence": 1,
    "tool_name": "Write",
    "file_path": "src/component.tsx",
    "input": {...},
    "result": "Success" | "Error: ...",
    "state_before": {
      "feature_id": 42,
      "passing_count": 10,
      "modified_files": ["src/component.tsx"]
    },
    "state_after": {
      "feature_id": 42,
      "passing_count": 10,
      "modified_files": ["src/component.tsx"]
    }
  }
}
```

**Benefits**:
- Resume from exact point of interruption
- Provides context for tool results
- Enables idempotent retries

#### D. Stale Work Detection & Recovery

**In `process_manager` or new recovery module**:
```python
def detect_stale_work(project_dir, timeout_minutes=60):
    """Find features stuck in in_progress state."""
    session = get_session()

    stuck = session.query(Feature).filter(
        Feature.in_progress == True,
        Feature.updated_at < datetime.now() - timedelta(minutes=timeout_minutes)
    ).all()

    return stuck

def recover_stale_work(project_dir):
    """
    On agent startup:
    1. Detect features in in_progress state for >60 minutes
    2. Load last session journal for that feature
    3. Check for blockers/errors
    4. Either reset or add to skip queue
    """
    stuck_features = detect_stale_work(project_dir)
    for feature in stuck_features:
        journal = load_session_journal(project_dir, feature.work_session_id)
        if journal.get("status") == "error":
            # Previous session errored - try again or skip
            feature.in_progress = False
            # Could implement escalation: add to queue with recovery_notes
        elif journal.get("blockers"):
            # Previous session had blockers - skip for now
            feature_skip(feature.id)
```

**Benefits**:
- Automatic recovery from crashes
- Prevents zombie work assignments
- Smart handling of blockers

#### E. Session Context Injection for Recovery

**Modify `prompts.py`** to include recovery context:
```python
def get_recovery_context(project_dir: Path) -> str:
    """Build context about previous work for recovery."""

    # Load last session journal
    last_session = load_latest_session_journal(project_dir)

    if not last_session:
        return ""

    context = f"""
## Recovery Context from Previous Session

**Previous Session Status**: {last_session.get('status')}
**Features Completed**: {last_session['final_stats']['passing']}
**Features Attempted**: {last_session.get('feature_id')}

**Previous Blockers**:
{json.dumps(last_session.get('blockers', []), indent=2)}

**Previous Errors**:
{json.dumps(last_session.get('errors', []), indent=2)}

**Decisions Made**:
{json.dumps(last_session.get('decisions', []), indent=2)}

**Session Duration**: {(last_session['ended_at'] - last_session['started_at']).total_seconds()} seconds
"""

    return context

# Inject into prompt:
prompt = get_coding_prompt_with_context(project_dir)
if is_recovery_session:
    recovery_context = get_recovery_context(project_dir)
    prompt = recovery_context + "\n\n" + prompt
```

**Benefits**:
- Agent learns from previous mistakes
- Avoids trying same failed approaches
- Can handle blockers more intelligently
- Provides continuity to human observers

---

### 5. IMPLEMENTATION PRIORITY ROADMAP

#### Phase 1: Foundation (Session Journal)
- Create `.session_journal/` with JSON logging
- Log all messages and tool uses
- Track feature stats at start/end
- **Effort**: 2-3 days | **Impact**: High (enables all other phases)

#### Phase 2: Feature Work Metadata
- Add `last_worked_on`, `work_session_id`, `attempt_count`, `last_error` to Feature model
- Track updates on each mark_in_progress/mark_passing call
- Add stale work detection
- **Effort**: 2-3 days | **Impact**: High (enables recovery)

#### Phase 3: Recovery Context
- Inject last session summary into next agent prompt
- Analyze blockers and errors automatically
- Skip features intelligently based on history
- **Effort**: 2-3 days | **Impact**: Medium (improves convergence)

#### Phase 4: Intelligent Resumption
- Detect stuck features on startup
- Clear stale in_progress flags with timeout
- Surface recovery notes to agent
- **Effort**: 2-3 days | **Impact**: Medium (operational stability)

#### Phase 5: Advanced Features (Optional)
- Message history reconstruction from journal
- Tool result caching for idempotent execution
- Automated root cause analysis of failures
- **Effort**: 3-4 days | **Impact**: Low (nice to have)

---

### 6. EXISTING PATTERNS TO FOLLOW

✓ **Database migrations**: Use SQLAlchemy migrations for new columns
✓ **File organization**: Follow `.project_dir` structure convention
✓ **JSON serialization**: Use datetime.isoformat() for timestamps
✓ **Error handling**: Log to session journal; don't crash on missing data
✓ **Async safety**: Use asyncio.Lock if accessed from multiple tasks
✓ **Recovery**: Follow `process_manager` pattern of graceful cleanup

---

### 7. KEY FILES TO MODIFY

1. **`api/database.py`** - Add feature metadata columns
2. **`mcp_server/feature_mcp.py`** - Update to set metadata on mutations
3. **`agent.py`** - Create session journal on startup/shutdown
4. **`prompts.py`** - Add recovery context injection
5. **`progress.py`** - Add session journal logging
6. **NEW**: `session_recovery.py` - Recovery module (stale detection, context building)

---

### 8. CRITICAL INSIGHTS

- **No message history exists**: Each session is stateless regarding conversation - this is the biggest gap
- **Feature state is solid**: Database design supports tracking work well
- **Process management works**: Lock files and PID tracking prevent conflicts
- **Recovery untested**: Current system would crash and lose context on agent failure
- **Blockers are invisible**: No mechanism to track why features are skipped
- **Decisions are lost**: Agent discoveries about requirements/design aren't recorded

The system is well-architected for **feature state tracking** but lacks **session/decision tracking** for recovery and learning. The three essential additions are: (1) session journals, (2) feature work metadata, and (3) recovery context injection.

agentId: a9a5626 (for resuming to continue this agent's work if needed)

Prompt:
Explore the Autocoder 2.0 codebase to understand how to implement session recovery/checkpointing:

1. **Current session state**: Look at `agent.py`, `progress.py`, and `server/services/process_manager.py` to understand what state exists during an agent session and where it's stored

2. **Process lifecycle**: Examine how agent sessions are started, stopped, and tracked - specifically the lock file mechanism and PID tracking in `process_manager.py`

3. **Feature state management**: Look at how feature progress is currently persisted (database writes, in_progress flag) and at what points we could create checkpoints

4. **MCP server state**: Examine `mcp_server/feature_mcp.py` to see how the MCP tools track and resume work on features

5. **Existing persistence patterns**: Find where the system already saves state to disk (database, lock files, config) and what patterns we can follow

Focus on identifying what state would need to be saved for recovery and the best checkpoint triggers (after tool use, after feature completion, etc.).


Explore git integration opportunities

Prompt:
Explore the Autocoder 2.0 codebase to understand how to implement git integration:

1. **Existing git usage**: Look for any current git operations in the codebase (bash commands, git status checks, etc.)

2. **Security model**: Examine `security.py` to understand the bash command restrictions and how git commands would need to be allowed

3. **Project structure**: Look at how projects are organized and where git repositories would typically be located relative to project directories

4. **Feature completion hooks**: Find where features are marked as passing (`feature_mark_passing`) to identify integration points for auto-commits

5. **Session boundaries**: Look at session start/end patterns where git operations like branch creation or PR creation could be triggered

6. **Process management**: Understand how git operations would fit into the process management model (subprocess execution, output capturing)

Focus on finding the natural integration points for git operations and any existing patterns for running external commands.


Perfect! Now I have a comprehensive view. Let me create a summary report of my findings:

## COMPREHENSIVE GIT INTEGRATION EXPLORATION REPORT

Based on my thorough analysis of the Autocoder 2.0 codebase, here are the key findings for implementing git integration:

---

### 1. EXISTING GIT USAGE IN THE CODEBASE

**Git is already extensively integrated!** The codebase has:

- **File**: `C:\Users\MikeSwanson\claude-projects\Autocode-remix\Autocoder-2.0\server\services\git_operations.py` (575 lines)
  - Full async git wrapper using `asyncio.create_subprocess_exec`
  - Supports: status, diff, log, add, commit, push, pull, fetch, checkout, branch creation, stash, reset
  - Returns structured data: `GitStatus`, `GitCommit`, `GitResult` classes
  - Uses semver approach with hash extraction via regex

- **File**: `C:\Users\MikeSwanson\claude-projects\Autocode-remix\Autocoder-2.0\server\agents\git_agent.py` (406 lines)
  - Dedicated `GitAgent` class orchestrating version control
  - Tracks changes: `ChangeEntry` records which files changed, by which agent, for which feature
  - **Key methods**:
    - `track_change()`: Called by agents to log file modifications
    - `on_feature_complete()`: Auto-commits when feature passes
    - `on_session_end()`: Commits remaining changes at session end
    - `provide_review_context()`: Pulls latest and provides diffs for code review
    - `handle_conflict()`: Analyzes merge conflicts
  - Configurable: conventional/simple/detailed commit styles
  - Auto-push on: `["feature_complete", "session_end"]`

- **File**: `C:\Users\MikeSwanson\claude-projects\Autocode-remix\Autocoder-2.0\server\routers\git.py` (490 lines)
  - FastAPI REST endpoints for git operations
  - WebSocket endpoint for real-time git status updates
  - All CRUD operations exposed: `/api/git/projects/{name}/status`, `/commit`, `/push`, `/pull`, etc.

---

### 2. SECURITY MODEL - BASH COMMAND RESTRICTIONS

**File**: `C:\Users\MikeSwanson\claude-projects\Autocode-remix\Autocoder-2.0\security.py` (442 lines)

**Git commands are ALREADY ALLOWED:**
- Line 65: `"git"` is in `ALLOWED_COMMANDS` set
- Line 66: `"gh"` (GitHub CLI) is also allowed

**Security approach:**
- Uses an **allowlist** (whitelist) - only explicitly permitted commands can run
- `extract_commands()` function parses compound commands (pipes, `&&`, `||`, `;`)
- Special validation for sensitive commands: `pkill`, `chmod`, `init.sh`
- Git operations don't require extra validation beyond being in the allowlist

**Integration point**: The `bash_security_hook()` async function validates all Bash tool use before execution

---

### 3. PROJECT STRUCTURE & GIT REPOSITORY LOCATIONS

**Architecture**:
```
Project Directory (git repo root)
├── features.db (SQLite: feature tracking)
├── feature_graph.db (SQLite: feature dependencies & file locks)
├── .agent.lock (process lock file)
├── server/
│   ├── services/
│   │   ├── git_operations.py
│   │   └── feature_graph.py
│   ├── agents/
│   │   └── git_agent.py
│   └── routers/
│       └── git.py
└── [project source files]
```

**Git structure**: Each project is expected to be a git repository. Git operations work on `project_dir` as the working directory:
- File: `git_operations.py`, line 92: `cwd=str(self.project_dir)`
- Projects resolved via registry: `get_project_path(project_name)`

---

### 4. FEATURE COMPLETION HOOKS - NATURAL INTEGRATION POINTS

**Primary integration point**: `on_feature_complete(feature_id: int, feature_name: Optional[str])`

**File**: `C:\Users\MikeSwanson\claude-projects\Autocode-remix\Autocoder-2.0\server\agents\git_agent.py`, lines 116-178

**Flow**:
1. When feature marked passing (via `feature_mark_passing` MCP tool)
2. `ParallelAgentRunner.run_single_feature()` calls `await self.git_agent.on_feature_complete(feature.id, feature.name)` (line 255)
3. Git agent automatically:
   - Collects all tracked changes for that feature
   - Generates commit message (conventional format: `feat: implement feature #ID`)
   - Stages & commits files
   - Optionally auto-pushes if `"feature_complete" in push_on` (line 170)

**Tracking**: Changes tracked via `track_change()` which populates `change_log: list[ChangeEntry]`
- Each entry: file, agent_id, feature_id, change_type, timestamp, summary

---

### 5. SESSION BOUNDARIES - AUTO-COMMIT POINTS

**Session end hook**: `on_session_end()` method

**File**: `C:\Users\MikeSwanson\claude-projects\Autocode-remix\Autocoder-2.0\server\agents\git_agent.py`, lines 180-215

**Triggers**:
- Called from `ParallelAgentRunner.run_parallel()` when loop ends (line 255 in parallel_agent.py)
- Creates "WIP" commit message if multiple features in flight
- Auto-pushes if `"session_end" in push_on` config

**Config location**: `C:\Users\MikeSwanson\claude-projects\Autocode-remix\Autocoder-2.0\config.py`, lines 22-29

```python
@dataclass
class GitAgentConfig:
    auto_commit: bool = True
    auto_push: bool = False
    push_on: list[str] = ["feature_complete", "session_end"]
    pull_before: list[str] = ["code_review", "feature_start"]
```

---

### 6. PROCESS MANAGEMENT - SUBPROCESS EXECUTION MODEL

**File**: `C:\Users\MikeSwanson\claude-projects\Autocode-remix\Autocoder-2.0\server\services\process_manager.py` (553 lines)

**Key patterns for git operations integration**:

1. **Async subprocess execution** (lines 89-101 in git_operations.py):
   ```python
   process = await asyncio.create_subprocess_exec(
       *cmd,
       cwd=str(self.project_dir),
       stdout=asyncio.subprocess.PIPE,
       stderr=asyncio.subprocess.PIPE,
   )
   stdout, stderr = await asyncio.wait_for(
       process.communicate(),
       timeout=timeout,
   )
   ```

2. **Output capturing & sanitization** (process_manager.py):
   - Lines 23-37: Sensitive data patterns (API keys, tokens, passwords)
   - Lines 40-44: `sanitize_output()` redacts before logging
   - Line 198: UTF-8 decode with error handling

3. **Callback pattern for real-time updates** (process_manager.py, lines 118-136):
   - Multiple output callbacks registered
   - Status callbacks for state changes
   - Async-safe via `_callbacks_lock`

4. **Cross-platform support**:
   - Uses `psutil` for process management
   - Process tree killing (recursive children)
   - Lock files to prevent duplicate agents

---

### 7. NATURAL INTEGRATION PATTERNS ALREADY ESTABLISHED

#### A. Git Agent Initialization
**File**: `parallel_agent.py`, lines 114-121
```python
self.git_agent = GitAgent(
    project_dir,
    GitAgentConfig(
        auto_commit=True,
        auto_push=False,
        commit_style="conventional",
    ),
)
```

#### B. Change Tracking During Coding
**File**: `parallel_agent.py`, lines 213-218
```python
for file in files_touched:
    await self.git_agent.track_change(
        file=file,
        agent_id=agent_id,
        feature_id=feature.id,
        change_type="modified",
    )
```

#### C. Feature Completion Hook
**File**: `parallel_agent.py`, line 255
```python
await self.git_agent.on_feature_complete(feature.id, feature.name)
```

#### D. Session End Hook
**File**: `parallel_agent.py`, line 256
```python
await self.git_agent.on_session_end()
```

#### E. REST API Endpoints
- `GET /api/git/projects/{project_name}/status` - repo status
- `POST /api/git/projects/{project_name}/commit` - create commit
- `POST /api/git/projects/{project_name}/push` - push changes
- `WebSocket /api/git/projects/{project_name}/ws` - real-time updates

---

### 8. FEATURE GRAPH & FILE LOCKING

**File**: `C:\Users\MikeSwanson\claude-projects\Autocode-remix\Autocoder-2.0\server\services\feature_graph.py` (493 lines)

Tracks:
- Files touched by each feature (prevents parallel conflicts)
- Feature dependencies
- Complexity estimates
- Current file locks by feature_id

**Integration with git**: The GitAgent can access file conflict info via:
- `handle_conflict()` method detects merge markers
- `_can_auto_resolve()` checks for lock files, .mod, .sum files

---

### 9. MCP TOOL - FEATURE COMPLETION TRIGGER

**File**: `C:\Users\MikeSwanson\claude-projects\Autocode-remix\Autocoder-2.0\mcp_server\feature_mcp.py`

**The MCP tool `feature_mark_passing` is the event trigger for git commits:**
- Called by coding agents when feature complete
- Updates database: `Feature.passes = 1`
- **This is where git integration should be triggered**

---

## RECOMMENDED GIT INTEGRATION POINTS

### Priority 1 (Already Implemented)
1. ✅ Git command allowlist in security.py
2. ✅ GitOperations async wrapper
3. ✅ GitAgent orchestration
4. ✅ REST API endpoints
5. ✅ WebSocket real-time updates

### Priority 2 (Ready to Use)
1. Feature completion auto-commit workflow
2. Session end auto-commit
3. Change tracking via agents
4. Conflict detection and analysis

### Priority 3 (Future Enhancements)
1. Branch creation per feature
2. PR creation via GitHub API (gh CLI already allowed)
3. Automatic code review pulls
4. Merge conflict auto-resolution for lock files

---

## KEY FILES FOR INTEGRATION

| File | Purpose | Key Classes |
|------|---------|-------------|
| git_operations.py | Low-level git commands | `GitOperations`, `GitStatus`, `GitResult` |
| git_agent.py | Orchestration | `GitAgent`, `ChangeEntry`, `GitAgentConfig` |
| git.py | REST API | FastAPI routers with Pydantic models |
| security.py | Command validation | `bash_security_hook`, `ALLOWED_COMMANDS` |
| parallel_agent.py | Integration point | Calls git_agent methods on completion |
| process_manager.py | Process lifecycle | Subprocess execution, output sanitization |
| config.py | Configuration | `GitAgentConfig` with push_on/pull_before |
| feature_graph.py | File conflict tracking | File locks, dependencies |

---

## SUMMARY

The Autocoder 2.0 codebase has **comprehensive git integration already built-in**. The architecture naturally supports:

1. **Automatic commits** on feature completion
2. **Session-end commits** for work-in-progress
3. **Change tracking** by agent and feature
4. **Real-time git status** via WebSocket
5. **Conflict detection** and analysis
6. **Configurable push strategy** (auto-push on feature_complete/session_end)
7. **Security-validated** git command execution
8. **File locking** to prevent parallel conflicts

The main integration points are:
- **Git operations**: Async subprocess execution with output capturing
- **Feature completion**: Auto-commit hook in `on_feature_complete()`
- **Session boundaries**: Auto-commit on `on_session_end()`
- **Security**: Git already in allowlist, no special privileges needed
- **REST API**: Full CRUD operations exposed via FastAPI

This is a production-ready, well-architected implementation that requires minimal additional work to achieve the project's git workflow goals.

agentId: ad35e39 (for resuming to continue this agent's work if needed)

Prompt:
Explore the Autocoder 2.0 codebase to understand how to implement git integration:

1. **Existing git usage**: Look for any current git operations in the codebase (bash commands, git status checks, etc.)

2. **Security model**: Examine `security.py` to understand the bash command restrictions and how git commands would need to be allowed

3. **Project structure**: Look at how projects are organized and where git repositories would typically be located relative to project directories

4. **Feature completion hooks**: Find where features are marked as passing (`feature_mark_passing`) to identify integration points for auto-commits

5. **Session boundaries**: Look at session start/end patterns where git operations like branch creation or PR creation could be triggered

6. **Process management**: Understand how git operations would fit into the process management model (subprocess execution, output capturing)

Focus on finding the natural integration points for git operations and any existing patterns for running external commands.


Now I'll gather clarifications from you to ensure my implementation plan aligns with your specific needs:


AskUserQuestion

Question: I found that Autocoder 2.0 already has extensive git integration built-in (git_agent.py, git_operations.py, auto-commits on feature completion). Do you want me to enhance the existing git integration or implement it from scratch?

Choices:
Build on the existing GitAgent and git_operations.py, adding features like PR creation, better branch management, and improved conflict handling.
Ignore existing git code and implement a new git integration system from scratch based on the original requirements.
Let me see what's already implemented before deciding whether to enhance or replace it.

Question: For the multi-model support (Opus/Sonnet/Haiku), what approach do you prefer?

Choices:
Use configuration rules to route tasks to different models based on complexity, keywords, or task type.
Have an AI orchestrator analyze each task and decide which model to use.
Start with Haiku, escalate to Sonnet if it struggles, escalate to Opus for complex decisions.
Add UI controls to let users choose which model to use for each session or feature.

Question: For the learning system that builds knowledge from past work, what scope of implementation do you want?

Choices:
Save successful implementations as searchable markdown/text. Simple keyword search over saved solutions.
Use embeddings and vector database for sophisticated similarity matching of past solutions.
Have AI analyze completed features and extract reusable patterns, storing them as structured knowledge.

Question: Which enhancements should I prioritize in the implementation plan?

Choices:
Real-time token usage display, cost estimation, session history. You mentioned this is highest priority.
Resume interrupted sessions, save agent context, detect stuck states. You mentioned this is highest priority.
Better handling of stuck/failed features, model escalation, skip mechanisms.
Route tasks to appropriate models (Opus/Sonnet/Haiku) based on complexity.


Answer questions?