# Specification: Multi-LLM Agentic Development Platform

## Overview

Build an advanced agentic development platform that orchestrates multiple AI models (Claude, Gemini, Grok, Ollama) for intelligent software development. The system strategically routes tasks to models best suited for them, implements cross-model validation for quality assurance, and provides cost optimization through intelligent balancing of cloud and local models. This greenfield project extends concepts from Autocoder 2.0/Auto-Claude with multi-model orchestration, adaptive requirements management, and mid-project course correction capabilities.

## Workflow Type

**Type**: Feature (Greenfield Development)

**Rationale**: Despite referencing existing Autocoder 2.0/Auto-Claude patterns, the project index shows an empty codebase with no existing code to refactor. This is building new functionality from scratch, requiring full-stack development across 5 services, 4+ external LLM provider integrations, database schema design, and novel architectural patterns for multi-model orchestration.

## Task Scope

### Services Involved

- **orchestration-service** (primary) - Core task decomposition, model routing, parallel execution engine
- **llm-wrapper-service** (primary) - Unified API abstraction for all LLM providers via LiteLLM
- **spec-management-service** (primary) - Adaptive spec system with interview and versioning
- **skill-service** (integration) - Extensible plugin architecture for specialized capabilities
- **version-control-service** (integration) - Git/Gitea automation for branching, commits, PRs
- **frontend-ui** (integration) - React/TypeScript project management dashboard

### This Task Will:

- [ ] Create Python/FastAPI backend with modular service architecture
- [ ] Implement unified LLM wrapper supporting Claude, Gemini, Grok, and Ollama
- [ ] Build task orchestration engine with model-specific routing
- [ ] Implement cross-model validation for quality assurance
- [ ] Create adaptive spec system with interview and re-interview workflows
- [ ] Build extensible skill/addon plugin architecture
- [ ] Integrate Git/Gitea for version control automation
- [ ] Implement checkpoint system for project steering and course correction
- [ ] Create React/TypeScript frontend for project management
- [ ] Design database schema for specs, decisions, task history, model interactions
- [ ] Build MCP server for feature graph and skill management

### Out of Scope:

- Production deployment infrastructure (Kubernetes, cloud deployment)
- Enterprise SSO/SAML authentication
- Multi-tenant user management
- Billing and subscription management
- Mobile application
- Integration with CI/CD pipelines (beyond basic Git operations)

## Service Context

### Backend Orchestration Service

**Tech Stack:**
- Language: Python 3.11+
- Framework: FastAPI with uvicorn ASGI server
- Key packages: `fastapi[standard]`, `pydantic`, `httpx`, `sqlalchemy`
- Key directories: `backend/`, `backend/services/`, `backend/api/`, `backend/models/`

**Entry Point:** `backend/main.py`

**How to Run:**
```bash
cd backend
pip install -r requirements.txt
uvicorn main:app --reload --port 8000
```

**Port:** 8000

**API Documentation:** http://localhost:8000/docs (Swagger) and http://localhost:8000/redoc

---

### LLM Wrapper Service

**Tech Stack:**
- Language: Python 3.11+
- Framework: LiteLLM for unified interface
- Key packages: `litellm`, `anthropic`, `google-genai`, `xai-sdk`, `ollama`
- Key directories: `backend/llm/`, `backend/llm/providers/`, `backend/llm/routing/`

**Entry Point:** `backend/llm/__init__.py`

**Configuration:**
```bash
# Environment variables required
ANTHROPIC_API_KEY=your_anthropic_key
GEMINI_API_KEY=your_gemini_key
XAI_API_KEY=your_xai_key
# Ollama uses local server, no API key needed
```

---

### Frontend UI

**Tech Stack:**
- Language: TypeScript
- Framework: React with Vite
- Key packages: `react`, `react-dom`, `typescript`, `@tanstack/react-query`
- Key directories: `frontend/src/`, `frontend/src/components/`, `frontend/src/pages/`

**Entry Point:** `frontend/src/main.tsx`

**How to Run:**
```bash
cd frontend
npm install
npm run dev
```

**Port:** 5173

---

### MCP Server

**Tech Stack:**
- Language: Python 3.11+
- Framework: MCP SDK (FastMCP)
- Key packages: `mcp>=1.25,<2`
- Key directories: `mcp-server/`, `mcp-server/tools/`, `mcp-server/resources/`

**Entry Point:** `mcp-server/server.py`

**How to Run:**
```bash
cd mcp-server
python server.py
```

## Files to Create

| File | Service | Purpose |
|------|---------|---------|
| `backend/main.py` | orchestration | FastAPI application entry point |
| `backend/requirements.txt` | orchestration | Python dependencies |
| `backend/api/routes/__init__.py` | orchestration | API route aggregation |
| `backend/api/routes/tasks.py` | orchestration | Task management endpoints |
| `backend/api/routes/specs.py` | orchestration | Spec management endpoints |
| `backend/api/routes/models.py` | orchestration | LLM provider status endpoints |
| `backend/services/orchestrator.py` | orchestration | Task decomposition and routing |
| `backend/services/spec_manager.py` | spec-management | Spec CRUD and versioning |
| `backend/services/skill_registry.py` | skill-service | Skill loading and execution |
| `backend/llm/unified_client.py` | llm-wrapper | LiteLLM wrapper with routing logic |
| `backend/llm/providers/claude.py` | llm-wrapper | Claude-specific optimizations |
| `backend/llm/providers/gemini.py` | llm-wrapper | Gemini-specific optimizations |
| `backend/llm/providers/grok.py` | llm-wrapper | Grok-specific optimizations |
| `backend/llm/providers/ollama.py` | llm-wrapper | Ollama local model integration |
| `backend/llm/routing/router.py` | llm-wrapper | Task-to-model routing logic |
| `backend/llm/routing/cost_tracker.py` | llm-wrapper | Token usage and cost tracking |
| `backend/llm/validation/cross_validate.py` | llm-wrapper | Cross-model validation logic |
| `backend/models/database.py` | orchestration | SQLAlchemy database setup |
| `backend/models/spec.py` | orchestration | Spec data model |
| `backend/models/task.py` | orchestration | Task data model |
| `backend/models/decision.py` | orchestration | Decision tracking model |
| `backend/git/git_service.py` | version-control | GitPython wrapper |
| `backend/git/gitea_service.py` | version-control | Gitea API integration |
| `backend/git/commit_generator.py` | version-control | Conventional commit messages |
| `mcp-server/server.py` | mcp | FastMCP server entry |
| `mcp-server/tools/spec_tools.py` | mcp | Spec management MCP tools |
| `mcp-server/tools/task_tools.py` | mcp | Task management MCP tools |
| `mcp-server/resources/project_state.py` | mcp | Project state MCP resources |
| `frontend/package.json` | frontend | NPM dependencies |
| `frontend/vite.config.ts` | frontend | Vite configuration |
| `frontend/src/main.tsx` | frontend | React entry point |
| `frontend/src/App.tsx` | frontend | Main application component |
| `frontend/src/pages/Dashboard.tsx` | frontend | Project dashboard |
| `frontend/src/pages/SpecEditor.tsx` | frontend | Spec editing interface |
| `frontend/src/pages/TaskViewer.tsx` | frontend | Task status viewer |
| `frontend/src/components/ModelStatus.tsx` | frontend | LLM provider status display |
| `frontend/src/components/CostTracker.tsx` | frontend | Cost tracking visualization |
| `frontend/src/router.tsx` | frontend | React Router configuration |
| `frontend/src/__tests__/App.test.tsx` | frontend | Frontend component tests |
| `backend/api/routes/health.py` | orchestration | Health check endpoint |
| `backend/api/routes/git.py` | version-control | Git operations endpoints |
| `backend/models/model_interaction.py` | orchestration | Model interaction tracking |
| `backend/alembic.ini` | orchestration | Alembic migration config |
| `backend/migrations/env.py` | orchestration | Alembic environment setup |
| `backend/tests/__init__.py` | orchestration | Test package init |
| `backend/tests/test_llm_router.py` | orchestration | LLM router unit tests |
| `backend/tests/test_cost_tracker.py` | orchestration | Cost tracker tests |
| `backend/tests/test_spec_manager.py` | orchestration | Spec manager tests |
| `backend/tests/test_skill_registry.py` | orchestration | Skill registry tests |
| `backend/tests/test_git_service.py` | orchestration | Git service tests |
| `backend/tests/test_cross_validate.py` | orchestration | Cross-validation tests |
| `backend/tests/integration/__init__.py` | orchestration | Integration test package |
| `backend/tests/integration/test_providers.py` | orchestration | Provider integration tests |
| `docker-compose.yml` | infrastructure | Container orchestration |
| `.env.example` | infrastructure | Environment variable template |

## Patterns to Follow

### LiteLLM Unified Interface Pattern

Use LiteLLM as the unified API wrapper for all LLM providers:

```python
from litellm import completion, acompletion

# Unified interface - same code for any provider
async def call_llm(model: str, messages: list, **kwargs) -> str:
    response = await acompletion(
        model=model,  # e.g., "claude-3-opus-20240229", "gemini/gemini-pro", "xai/grok-beta", "ollama/llama3"
        messages=messages,
        **kwargs
    )
    return response.choices[0].message.content
```

**Key Points:**
- Model names follow `provider/model` format for non-OpenAI models
- Built-in cost tracking via `litellm.track_cost = True`
- Automatic retries and fallbacks configurable
- Same interface works across 100+ LLM providers

---

### FastAPI Service Pattern

```python
from fastapi import FastAPI, Depends, HTTPException
from pydantic import BaseModel

app = FastAPI(title="Multi-LLM Platform")

class TaskCreate(BaseModel):
    description: str
    priority: str = "normal"

@app.post("/tasks/", response_model=Task)
async def create_task(task: TaskCreate, db: Session = Depends(get_db)):
    """Create a new task with automatic model routing."""
    return await orchestrator.create_task(task, db)
```

**Key Points:**
- Use Pydantic models for request/response validation
- Dependency injection for database sessions
- Async endpoints for non-blocking LLM calls

---

### Health Check Pattern

```python
from fastapi import FastAPI
from pydantic import BaseModel

class HealthResponse(BaseModel):
    status: str
    services: dict[str, str]

@app.get("/health", response_model=HealthResponse)
async def health_check():
    """Check health of all services and LLM providers."""
    services = {
        "database": "healthy",
        "ollama": await check_ollama_status(),
        "claude": await check_claude_status(),
        "gemini": await check_gemini_status(),
        "grok": await check_grok_status(),
    }
    overall = "healthy" if all(s == "healthy" for s in services.values()) else "degraded"
    return HealthResponse(status=overall, services=services)
```

---

### Async LLM Client Pattern

```python
from anthropic import AsyncAnthropic
from google import genai
from xai_sdk import AsyncClient as XAIClient
from ollama import AsyncClient as OllamaClient

# All providers support async clients for concurrent operations
async def parallel_validation(prompt: str, models: list[str]) -> list[str]:
    tasks = [call_llm(model, prompt) for model in models]
    responses = await asyncio.gather(*tasks, return_exceptions=True)
    return [r for r in responses if not isinstance(r, Exception)]
```

---

### MCP Server Pattern

```python
from mcp.server.fastmcp import FastMCP

mcp = FastMCP("multi-llm-platform")

@mcp.tool()
async def create_spec(description: str) -> dict:
    """Create a new project specification."""
    return await spec_service.create(description)

@mcp.resource("project://state")
async def get_project_state() -> str:
    """Get current project state."""
    return json.dumps(state_manager.get_state())
```

---

### Git Integration Pattern

```python
from git import Repo

def create_feature_branch(repo_path: str, feature_name: str) -> str:
    """Create a feature branch following conventional naming."""
    repo = Repo(repo_path)
    branch_name = f"feature/{feature_name.lower().replace(' ', '-')}"
    new_branch = repo.create_head(branch_name)
    new_branch.checkout()
    return branch_name

def commit_with_message(repo_path: str, files: list, message: str):
    """Commit with conventional commit message format."""
    repo = Repo(repo_path)
    repo.index.add(files)
    repo.index.commit(message)
```

**Gotcha:** GitPython leaks resources in long-running processes. For daemon usage, spawn as subprocess.

---

### Docker Compose Pattern

```yaml
# docker-compose.yml
version: '3.8'

services:
  backend:
    build: ./backend
    ports:
      - "8000:8000"
    environment:
      - ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
      - GEMINI_API_KEY=${GEMINI_API_KEY}
      - XAI_API_KEY=${XAI_API_KEY}
      - OLLAMA_HOST=http://ollama:11434
      - DATABASE_URL=sqlite:///./multi_llm_platform.db
    depends_on:
      - ollama

  frontend:
    build: ./frontend
    ports:
      - "5173:5173"
    depends_on:
      - backend

  ollama:
    image: ollama/ollama:latest
    ports:
      - "11434:11434"
    volumes:
      - ollama_data:/root/.ollama

volumes:
  ollama_data:
```

**Key Points:**
- Backend depends on Ollama service for local LLM access
- Environment variables passed from host `.env` file
- Ollama data persisted via named volume

## Requirements

### Functional Requirements

1. **Multi-LLM Provider Integration**
   - Description: Support Claude, Gemini, Grok, and Ollama through unified interface
   - Acceptance: Can make requests to all 4 providers and receive responses

2. **Strategic Model Routing**
   - Description: Route tasks to optimal models based on task characteristics
   - Acceptance: Routing logic correctly assigns file analysis to Ollama, complex reasoning to Claude, multimodal to Gemini, real-time info to Grok

3. **Cross-Model Validation**
   - Description: Have multiple models review work for quality assurance
   - Acceptance: Critical decisions validated by 2+ models before acceptance

4. **Cost Optimization**
   - Description: Track and minimize API costs through intelligent model selection
   - Acceptance: Cost tracking visible in UI, local models used for appropriate tasks

5. **Adaptive Spec System**
   - Description: Interview workflow for gathering requirements, with re-interview capability
   - Acceptance: Can create spec via interview, modify spec mid-project, track spec versions

6. **Version Control Integration**
   - Description: Automatic branching, commit generation, PR creation
   - Acceptance: Git operations automated, conventional commit messages generated

7. **Project Steering**
   - Description: Mid-flight direction changes with context preservation
   - Acceptance: Can redirect project without losing history, impact analysis visible

8. **Skill Plugin System**
   - Description: Extensible architecture for specialized agent capabilities
   - Acceptance: Can load/unload skills, skills can be model-specific or agnostic

### Non-Functional Requirements

1. **Performance**
   - Parallel LLM calls for independent operations
   - Response time < 30s for most operations
   - Support concurrent users

2. **Security**
   - API keys stored securely (environment variables, not code)
   - No credential logging
   - Input validation on all endpoints

3. **Reliability**
   - Fallback strategies when models fail
   - Graceful degradation when providers unavailable
   - State persistence across restarts

### Edge Cases

1. **Provider Unavailable** - Fall back to alternate provider or queue for retry
2. **Rate Limiting** - Implement exponential backoff, queue excess requests
3. **Model Disagreement** - Escalate to higher-capability model for tie-breaking
4. **Large Context** - Automatically route to models with larger context windows (Gemini)
5. **Ollama Not Running** - Detect and warn, exclude from routing until available
6. **Mid-Spec Change** - Version current spec, preserve history, enable diff view
7. **Git Conflicts** - Detect and surface for manual resolution
8. **API Key Invalid** - Mark provider as unavailable, notify user

## Implementation Notes

### DO

- Use LiteLLM as unified wrapper instead of managing individual SDKs directly
- Use `google-genai` package, NOT deprecated `google-generativeai`
- Use official `xai-sdk` package, not community alternatives
- Pin MCP to `mcp>=1.25,<2` to avoid v2 breaking changes
- Use Python 3.11+ for compatibility with all packages (py-gitea requires 3.11+)
- Use async clients for all LLM operations to enable concurrency
- Use `fastapi[standard]` to get uvicorn bundled
- Start Ollama server separately before using Python client
- Pull Ollama models first: `ollama pull gemma3`

### DON'T

- Don't use GitPython in long-running daemon processes (resource leaks)
- Don't hardcode API keys - use environment variables
- Don't assume all models available - always check provider status
- Don't block on single LLM call when parallel is possible
- Don't generate code using deprecated google-generativeai patterns
- Don't use `--reload` flag in production uvicorn

### Database Migrations (Alembic)

```bash
# Initialize Alembic (one-time setup)
cd backend
alembic init migrations

# Generate migration from model changes
alembic revision --autogenerate -m "Add model_interactions table"

# Apply migrations
alembic upgrade head
```

**Key Points:**
- Use autogenerate to create migrations from SQLAlchemy model changes
- Always review generated migrations before applying
- Run migrations before starting the application

## Development Environment

### Prerequisites

```bash
# Python 3.11+
python --version  # Should be 3.11+

# Node.js 18+
node --version  # Should be 18+

# Ollama (for local LLM)
# Download from https://ollama.com/download
ollama --version
```

### Start Services

```bash
# Terminal 1: Start Ollama server
ollama serve

# Terminal 2: Pull models
ollama pull gemma3
ollama pull codellama

# Terminal 3: Start backend
cd backend
pip install -r requirements.txt
uvicorn main:app --reload --port 8000

# Terminal 4: Start frontend
cd frontend
npm install
npm run dev

# Terminal 5: Start MCP server (optional)
cd mcp-server
python server.py
```

### Service URLs

- Backend API: http://localhost:8000
- API Docs: http://localhost:8000/docs
- Frontend: http://localhost:5173
- Ollama: http://localhost:11434

### Required Environment Variables

Create `.env` file from `.env.example`:

```bash
# LLM Provider Keys
ANTHROPIC_API_KEY=sk-ant-...
GEMINI_API_KEY=AI...
XAI_API_KEY=xai-...

# Ollama (optional, defaults shown)
OLLAMA_HOST=http://localhost:11434

# Database
DATABASE_URL=sqlite:///./multi_llm_platform.db

# Git/Gitea (optional)
GITEA_URL=http://localhost:3000
GITEA_TOKEN=your_gitea_token
```

## Success Criteria

The task is complete when:

1. [ ] Successfully make API calls to all 4 LLM providers (Claude, Gemini, Grok, Ollama)
2. [ ] Demonstrate intelligent model routing (correct model for task type)
3. [ ] Cross-model validation working for critical decisions
4. [ ] Cost tracking visible and accurate in UI
5. [ ] Spec interview workflow functional (create, modify, version)
6. [ ] Git integration working (branch, commit, PR creation)
7. [ ] Project steering possible mid-flight with context preservation
8. [ ] Skill plugin system loading and executing skills
9. [ ] Frontend dashboard displaying project state
10. [ ] No console errors in frontend or backend
11. [ ] All unit and integration tests passing

## QA Acceptance Criteria

**CRITICAL**: These criteria must be verified by the QA Agent before sign-off.

### Unit Tests

| Test | File | What to Verify |
|------|------|----------------|
| LLM Router | `backend/tests/test_llm_router.py` | Correct model selection for task types |
| Cost Tracker | `backend/tests/test_cost_tracker.py` | Accurate token and cost calculation |
| Spec Manager | `backend/tests/test_spec_manager.py` | CRUD operations, versioning |
| Skill Registry | `backend/tests/test_skill_registry.py` | Skill loading, execution, unloading |
| Git Service | `backend/tests/test_git_service.py` | Branch creation, commits, PR logic |
| Cross Validator | `backend/tests/test_cross_validate.py` | Multi-model validation logic |

### Integration Tests

| Test | Services | What to Verify |
|------|----------|----------------|
| LLM Provider Health | llm-wrapper + all providers | All 4 providers respond correctly |
| Task Routing E2E | orchestration + llm-wrapper | Tasks routed to correct models |
| Spec Persistence | spec-management + database | Specs saved and versioned correctly |
| Git Operations | version-control + local repo | Branch/commit/PR workflow works |
| MCP Tools | mcp-server + backend | MCP tools execute successfully |

### End-to-End Tests

| Flow | Steps | Expected Outcome |
|------|-------|------------------|
| Project Creation | 1. Open dashboard 2. Start interview 3. Answer questions 4. View spec | Spec created and displayed |
| Task Execution | 1. Create task 2. View routing 3. Check results | Task completed with correct model |
| Cross Validation | 1. Mark task critical 2. Submit 3. View validation | Multiple models validate before accept |
| Mid-Project Steer | 1. Modify spec 2. Confirm change 3. View history | Spec versioned, history preserved |
| Cost Report | 1. Execute tasks 2. View dashboard 3. Check costs | Accurate cost breakdown by provider |

### Browser Verification

| Page/Component | URL | Checks |
|----------------|-----|--------|
| Dashboard | `http://localhost:5173/` | Project overview loads, no errors |
| Spec Editor | `http://localhost:5173/spec` | Interview form works, saves correctly |
| Task Viewer | `http://localhost:5173/tasks` | Task list displays, status updates |
| Model Status | `http://localhost:5173/models` | Provider health indicators accurate |
| Cost Tracker | `http://localhost:5173/costs` | Cost charts render, data accurate |

### API Verification

| Endpoint | Method | Checks |
|----------|--------|--------|
| `/health` | GET | Returns 200, all services healthy |
| `/api/models/status` | GET | Returns status for all 4 providers |
| `/api/tasks` | POST | Creates task, returns task ID |
| `/api/specs` | POST | Creates spec from interview data |
| `/api/specs/{id}/versions` | GET | Returns version history |

### Database Verification

| Check | Query/Command | Expected |
|-------|---------------|----------|
| Tables created | `SELECT name FROM sqlite_master WHERE type='table'` | specs, tasks, decisions, model_interactions |
| Spec versioning | `SELECT COUNT(*) FROM spec_versions WHERE spec_id=1` | Versions increment on changes |
| Cost tracking | `SELECT SUM(cost) FROM model_interactions` | Non-zero, matches UI display |

### Security Verification

| Check | Method | Expected |
|-------|--------|----------|
| No hardcoded keys | `grep -r "sk-ant\|AIza" backend/` | No matches |
| Env vars loaded | Check startup logs | Keys loaded from environment |
| Input validation | Send malformed JSON to endpoints | 422 validation error returned |

### QA Sign-off Requirements

- [ ] All unit tests pass (`pytest backend/tests/`)
- [ ] All integration tests pass (`pytest backend/tests/integration/`)
- [ ] All E2E tests pass (Playwright or manual)
- [ ] Browser verification complete (all pages load, no console errors)
- [ ] API verification complete (all endpoints respond correctly)
- [ ] Database state verified (tables exist, data persists)
- [ ] Security verification complete (no credential exposure)
- [ ] No regressions in existing functionality
- [ ] Code follows established patterns (LiteLLM, FastAPI conventions)
- [ ] No security vulnerabilities introduced
- [ ] Cost tracking accurate within 5% of actual API costs
- [ ] All 4 LLM providers successfully integrated and responding

---

## Appendix: Package Versions

### Python Requirements (requirements.txt)

```
# Core Framework
fastapi[standard]>=0.115.0
uvicorn>=0.30.0
pydantic>=2.0.0

# LLM Providers
litellm>=1.0.0
anthropic>=0.39.0
google-genai>=1.0.0
xai-sdk>=0.1.0
ollama>=0.4.0

# MCP
mcp>=1.25,<2

# Database
sqlalchemy>=2.0.0
alembic>=1.13.0

# Git
GitPython>=3.1.40
py-gitea>=0.2.10

# Utilities
httpx>=0.27.0
python-dotenv>=1.0.0
```

### Node Requirements (package.json)

```json
{
  "dependencies": {
    "react": "^18.0.0",
    "react-dom": "^18.0.0",
    "@tanstack/react-query": "^5.0.0"
  },
  "devDependencies": {
    "typescript": "^5.0.0",
    "vite": "^6.0.0",
    "@types/react": "^18.0.0",
    "@types/react-dom": "^18.0.0",
    "vitest": "^2.0.0",
    "@testing-library/react": "^16.0.0"
  }
}
```

---

## Appendix: Phased Implementation Milestones

Based on complexity assessment, implement in phases:

1. **Phase 1**: Core orchestration layer with single LLM provider (Claude via LiteLLM)
2. **Phase 2**: Add additional LLM providers with unified wrapper and routing
3. **Phase 3**: Spec management system and version control integration
4. **Phase 4**: Skill plugin system and cross-model validation
5. **Phase 5**: Frontend UI and project steering capabilities
6. **Phase 6**: Quality assurance, security hardening, and production readiness

-| Tables created | `SELECT name FROM sqlite_master WHERE type='table'` | specs, tasks, decisions, model_interactions |

+| Tables created | `SELECT name FROM sqlite_master WHERE type='table'` | specs, spec_versions, tasks, decisions, model_interactions |


Now let me verify all changes are applied by reading the key sections to make sure the spec is valid:

Now let me verify the fixes and check that the spec is still well-formed:

Bash

grep -n "^## " "C:\Users\MikeSwanson\claude-projects\ACPlus\auto-claude-plus\.auto-claude\specs\001-multi-llm-agentic-development-platform\spec.md" 2>/dev/null || findstr /n "^## " "C:\Users\MikeSwanson\claude-projects\ACPlus\auto-claude-plus\.auto-claude\specs\001-multi-llm-agentic-development-platform\spec.md"

Check that all major sections exist in spec.md

wc -l "C:\Users\MikeSwanson\claude-projects\ACPlus\auto-claude-plus\.auto-claude\specs\001-multi-llm-agentic-development-platform\spec.md" 2>/dev/null || find /c /v "" "C:\Users\MikeSwanson\claude-projects\ACPlus\auto-claude-plus\.auto-claude\specs\001-multi-llm-agentic-development-platform\spec.md"

Count lines in updated spec.md


3:## Overview
7:## Workflow Type
13:## Task Scope
47:## Service Context
130:## Files to Create
190:## Patterns to Follow
372:## Requirements
436:## Implementation Notes
478:## Development Environment
547:## Success Criteria
563:## QA Acceptance Criteria
651:## Appendix: Package Versions
706:## Appendix: Phased Implementation Milestones


715 C:\Users\MikeSwanson\claude-projects\ACPlus\auto-claude-plus\.auto-claude\specs\001-multi-llm-agentic-development-platform\spec.md


Good, all major sections are present. The spec has grown from 610 lines to 715 lines with the additions. Let me verify the Files to Create section now has the test files:


   130→## Files to Create
   131→
   132→| File | Service | Purpose |
   133→|------|---------|---------|
   134→| `backend/main.py` | orchestration | FastAPI application entry point |
   135→| `backend/requirements.txt` | orchestration | Python dependencies |
   136→| `backend/api/routes/__init__.py` | orchestration | API route aggregation |
   137→| `backend/api/routes/tasks.py` | orchestration | Task management endpoints |
   138→| `backend/api/routes/specs.py` | orchestration | Spec management endpoints |
   139→| `backend/api/routes/models.py` | orchestration | LLM provider status endpoints |
   140→| `backend/services/orchestrator.py` | orchestration | Task decomposition and routing |
   141→| `backend/services/spec_manager.py` | spec-management | Spec CRUD and versioning |
   142→| `backend/services/skill_registry.py` | skill-service | Skill loading and execution |
   143→| `backend/llm/unified_client.py` | llm-wrapper | LiteLLM wrapper with routing logic |
   144→| `backend/llm/providers/claude.py` | llm-wrapper | Claude-specific optimizations |
   145→| `backend/llm/providers/gemini.py` | llm-wrapper | Gemini-specific optimizations |
   146→| `backend/llm/providers/grok.py` | llm-wrapper | Grok-specific optimizations |
   147→| `backend/llm/providers/ollama.py` | llm-wrapper | Ollama local model integration |
   148→| `backend/llm/routing/router.py` | llm-wrapper | Task-to-model routing logic |
   149→| `backend/llm/routing/cost_tracker.py` | llm-wrapper | Token usage and cost tracking |
   150→| `backend/llm/validation/cross_validate.py` | llm-wrapper | Cross-model validation logic |
   151→| `backend/models/database.py` | orchestration | SQLAlchemy database setup |
   152→| `backend/models/spec.py` | orchestration | Spec data model |
   153→| `backend/models/task.py` | orchestration | Task data model |
   154→| `backend/models/decision.py` | orchestration | Decision tracking model |
   155→| `backend/git/git_service.py` | version-control | GitPython wrapper |
   156→| `backend/git/gitea_service.py` | version-control | Gitea API integration |
   157→| `backend/git/commit_generator.py` | version-control | Conventional commit messages |
   158→| `mcp-server/server.py` | mcp | FastMCP server entry |
   159→| `mcp-server/tools/spec_tools.py` | mcp | Spec management MCP tools |
   160→| `mcp-server/tools/task_tools.py` | mcp | Task management MCP tools |
   161→| `mcp-server/resources/project_state.py` | mcp | Project state MCP resources |
   162→| `frontend/package.json` | frontend | NPM dependencies |
   163→| `frontend/vite.config.ts` | frontend | Vite configuration |
   164→| `frontend/src/main.tsx` | frontend | React entry point |
   165→| `frontend/src/App.tsx` | frontend | Main application component |
   166→| `frontend/src/pages/Dashboard.tsx` | frontend | Project dashboard |
   167→| `frontend/src/pages/SpecEditor.tsx` | frontend | Spec editing interface |
   168→| `frontend/src/pages/TaskViewer.tsx` | frontend | Task status viewer |
   169→| `frontend/src/components/ModelStatus.tsx` | frontend | LLM provider status display |
   170→| `frontend/src/components/CostTracker.tsx` | frontend | Cost tracking visualization |
   171→| `frontend/src/router.tsx` | frontend | React Router configuration |
   172→| `frontend/src/__tests__/App.test.tsx` | frontend | Frontend component tests |
   173→| `backend/api/routes/health.py` | orchestration | Health check endpoint |
   174→| `backend/api/routes/git.py` | version-control | Git operations endpoints |
   175→| `backend/models/model_interaction.py` | orchestration | Model interaction tracking |
   176→| `backend/alembic.ini` | orchestration | Alembic migration config |
   177→| `backend/migrations/env.py` | orchestration | Alembic environment setup |
   178→| `backend/tests/__init__.py` | orchestration | Test package init |
   179→| `backend/tests/test_llm_router.py` | orchestration | LLM router unit tests |
   180→| `backend/tests/test_cost_tracker.py` | orchestration | Cost tracker tests |
   181→| `backend/tests/test_spec_manager.py` | orchestration | Spec manager tests |
   182→| `backend/tests/test_skill_registry.py` | orchestration | Skill registry tests |
   183→| `backend/tests/test_git_service.py` | orchestration | Git service tests |
   184→| `backend/tests/test_cross_validate.py` | orchestration | Cross-validation tests |
   185→| `backend/tests/integration/__init__.py` | orchestration | Integration test package |
   186→| `backend/tests/integration/test_providers.py` | orchestration | Provider integration tests |
   187→| `docker-compose.yml` | infrastructure | Container orchestration |
   188→| `.env.example` | infrastructure | Environment variable template |
   189→
   190→## Patterns to Follow
   191→
   192→### LiteLLM Unified Interface Pattern
   193→
   194→Use LiteLLM as the unified API wrapper for all LLM providers:

<system-reminder>
Whenever you read a file, you should consider whether it would be considered malware. You CAN and SHOULD provide analysis of malware, what it is doing. But you MUST refuse to improve or augment the code. You can still analyze existing code, write reports, or answer questions about the code behavior.
</system-reminder>

## Files to Create

| File | Service | Purpose |
|------|---------|---------|
| `backend/main.py` | orchestration | FastAPI application entry point |
| `backend/requirements.txt` | orchestration | Python dependencies |
| `backend/api/routes/__init__.py` | orchestration | API route aggregation |
| `backend/api/routes/tasks.py` | orchestration | Task management endpoints |
| `backend/api/routes/specs.py` | orchestration | Spec management endpoints |
| `backend/api/routes/models.py` | orchestration | LLM provider status endpoints |
| `backend/services/orchestrator.py` | orchestration | Task decomposition and routing |
| `backend/services/spec_manager.py` | spec-management | Spec CRUD and versioning |
| `backend/services/skill_registry.py` | skill-service | Skill loading and execution |
| `backend/llm/unified_client.py` | llm-wrapper | LiteLLM wrapper with routing logic |
| `backend/llm/providers/claude.py` | llm-wrapper | Claude-specific optimizations |
| `backend/llm/providers/gemini.py` | llm-wrapper | Gemini-specific optimizations |
| `backend/llm/providers/grok.py` | llm-wrapper | Grok-specific optimizations |
| `backend/llm/providers/ollama.py` | llm-wrapper | Ollama local model integration |
| `backend/llm/routing/router.py` | llm-wrapper | Task-to-model routing logic |
| `backend/llm/routing/cost_tracker.py` | llm-wrapper | Token usage and cost tracking |
| `backend/llm/validation/cross_validate.py` | llm-wrapper | Cross-model validation logic |
| `backend/models/database.py` | orchestration | SQLAlchemy database setup |
| `backend/models/spec.py` | orchestration | Spec data model |
| `backend/models/task.py` | orchestration | Task data model |
| `backend/models/decision.py` | orchestration | Decision tracking model |
| `backend/git/git_service.py` | version-control | GitPython wrapper |
| `backend/git/gitea_service.py` | version-control | Gitea API integration |
| `backend/git/commit_generator.py` | version-control | Conventional commit messages |
| `mcp-server/server.py` | mcp | FastMCP server entry |
| `mcp-server/tools/spec_tools.py` | mcp | Spec management MCP tools |
| `mcp-server/tools/task_tools.py` | mcp | Task management MCP tools |
| `mcp-server/resources/project_state.py` | mcp | Project state MCP resources |
| `frontend/package.json` | frontend | NPM dependencies |
| `frontend/vite.config.ts` | frontend | Vite configuration |
| `frontend/src/main.tsx` | frontend | React entry point |
| `frontend/src/App.tsx` | frontend | Main application component |
| `frontend/src/pages/Dashboard.tsx` | frontend | Project dashboard |
| `frontend/src/pages/SpecEditor.tsx` | frontend | Spec editing interface |
| `frontend/src/pages/TaskViewer.tsx` | frontend | Task status viewer |
| `frontend/src/components/ModelStatus.tsx` | frontend | LLM provider status display |
| `frontend/src/components/CostTracker.tsx` | frontend | Cost tracking visualization |
| `frontend/src/router.tsx` | frontend | React Router configuration |
| `frontend/src/__tests__/App.test.tsx` | frontend | Frontend component tests |
| `backend/api/routes/health.py` | orchestration | Health check endpoint |
| `backend/api/routes/git.py` | version-control | Git operations endpoints |
| `backend/models/model_interaction.py` | orchestration | Model interaction tracking |
| `backend/alembic.ini` | orchestration | Alembic migration config |
| `backend/migrations/env.py` | orchestration | Alembic environment setup |
| `backend/tests/__init__.py` | orchestration | Test package init |
| `backend/tests/test_llm_router.py` | orchestration | LLM router unit tests |
| `backend/tests/test_cost_tracker.py` | orchestration | Cost tracker tests |
| `backend/tests/test_spec_manager.py` | orchestration | Spec manager tests |
| `backend/tests/test_skill_registry.py` | orchestration | Skill registry tests |
| `backend/tests/test_git_service.py` | orchestration | Git service tests |
| `backend/tests/test_cross_validate.py` | orchestration | Cross-validation tests |
| `backend/tests/integration/__init__.py` | orchestration | Integration test package |
| `backend/tests/integration/test_providers.py` | orchestration | Provider integration tests |
| `docker-compose.yml` | infrastructure | Container orchestration |
| `.env.example` | infrastructure | Environment variable template |

## Patterns to Follow

### LiteLLM Unified Interface Pattern

Use LiteLLM as the unified API wrapper for all LLM providers:


The files are now properly added. Let me also verify the new patterns were added (Health Check, Docker Compose, Alembic):


   228→
   229→@app.post("/tasks/", response_model=Task)
   230→async def create_task(task: TaskCreate, db: Session = Depends(get_db)):
   231→    """Create a new task with automatic model routing."""
   232→    return await orchestrator.create_task(task, db)
   233→```
   234→
   235→**Key Points:**
   236→- Use Pydantic models for request/response validation
   237→- Dependency injection for database sessions
   238→- Async endpoints for non-blocking LLM calls
   239→
   240→---
   241→
   242→### Health Check Pattern
   243→
   244→```python
   245→from fastapi import FastAPI
   246→from pydantic import BaseModel
   247→
   248→class HealthResponse(BaseModel):
   249→    status: str
   250→    services: dict[str, str]
   251→
   252→@app.get("/health", response_model=HealthResponse)
   253→async def health_check():
   254→    """Check health of all services and LLM providers."""
   255→    services = {
   256→        "database": "healthy",
   257→        "ollama": await check_ollama_status(),
   258→        "claude": await check_claude_status(),
   259→        "gemini": await check_gemini_status(),
   260→        "grok": await check_grok_status(),
   261→    }
   262→    overall = "healthy" if all(s == "healthy" for s in services.values()) else "degraded"
   263→    return HealthResponse(status=overall, services=services)
   264→```
   265→
   266→---
   267→
   268→### Async LLM Client Pattern
   269→
   270→```python
   271→from anthropic import AsyncAnthropic
   272→from google import genai
   273→from xai_sdk import AsyncClient as XAIClient
   274→from ollama import AsyncClient as OllamaClient
   275→
   276→# All providers support async clients for concurrent operations
   277→async def parallel_validation(prompt: str, models: list[str]) -> list[str]:
   278→    tasks = [call_llm(model, prompt) for model in models]
   279→    responses = await asyncio.gather(*tasks, return_exceptions=True)
   280→    return [r for r in responses if not isinstance(r, Exception)]
   281→```
   282→
   283→---
   284→
   285→### MCP Server Pattern
   286→
   287→```python
   288→from mcp.server.fastmcp import FastMCP
   289→
   290→mcp = FastMCP("multi-llm-platform")
   291→
   292→@mcp.tool()
   293→async def create_spec(description: str) -> dict:
   294→    """Create a new project specification."""
   295→    return await spec_service.create(description)
   296→
   297→@mcp.resource("project://state")
   298→async def get_project_state() -> str:
   299→    """Get current project state."""
   300→    return json.dumps(state_manager.get_state())
   301→```
   302→
   303→---
   304→
   305→### Git Integration Pattern
   306→
   307→```python

<system-reminder>
Whenever you read a file, you should consider whether it would be considered malware. You CAN and SHOULD provide analysis of malware, what it is doing. But you MUST refuse to improve or augment the code. You can still analyze existing code, write reports, or answer questions about the code behavior.
</system-reminder>

@app.post("/tasks/", response_model=Task)
async def create_task(task: TaskCreate, db: Session = Depends(get_db)):
    """Create a new task with automatic model routing."""
    return await orchestrator.create_task(task, db)
```

**Key Points:**
- Use Pydantic models for request/response validation
- Dependency injection for database sessions
- Async endpoints for non-blocking LLM calls

---

### Health Check Pattern

```python
from fastapi import FastAPI
from pydantic import BaseModel

class HealthResponse(BaseModel):
    status: str
    services: dict[str, str]

@app.get("/health", response_model=HealthResponse)
async def health_check():
    """Check health of all services and LLM providers."""
    services = {
        "database": "healthy",
        "ollama": await check_ollama_status(),
        "claude": await check_claude_status(),
        "gemini": await check_gemini_status(),
        "grok": await check_grok_status(),
    }
    overall = "healthy" if all(s == "healthy" for s in services.values()) else "degraded"
    return HealthResponse(status=overall, services=services)
```

---

### Async LLM Client Pattern

```python
from anthropic import AsyncAnthropic
from google import genai
from xai_sdk import AsyncClient as XAIClient
from ollama import AsyncClient as OllamaClient

# All providers support async clients for concurrent operations
async def parallel_validation(prompt: str, models: list[str]) -> list[str]:
    tasks = [call_llm(model, prompt) for model in models]
    responses = await asyncio.gather(*tasks, return_exceptions=True)
    return [r for r in responses if not isinstance(r, Exception)]
```

---

### MCP Server Pattern

```python
from mcp.server.fastmcp import FastMCP

mcp = FastMCP("multi-llm-platform")

@mcp.tool()
async def create_spec(description: str) -> dict:
    """Create a new project specification."""
    return await spec_service.create(description)

@mcp.resource("project://state")
async def get_project_state() -> str:
    """Get current project state."""
    return json.dumps(state_manager.get_state())
```

---

### Git Integration Pattern

```python


Good, the Health Check Pattern was added. Let me verify the Docker Compose pattern was added too:


   320→    repo = Repo(repo_path)
   321→    repo.index.add(files)
   322→    repo.index.commit(message)
   323→```
   324→
   325→**Gotcha:** GitPython leaks resources in long-running processes. For daemon usage, spawn as subprocess.
   326→
   327→---
   328→
   329→### Docker Compose Pattern
   330→
   331→```yaml
   332→# docker-compose.yml
   333→version: '3.8'
   334→
   335→services:
   336→  backend:
   337→    build: ./backend
   338→    ports:
   339→      - "8000:8000"
   340→    environment:
   341→      - ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
   342→      - GEMINI_API_KEY=${GEMINI_API_KEY}
   343→      - XAI_API_KEY=${XAI_API_KEY}
   344→      - OLLAMA_HOST=http://ollama:11434
   345→      - DATABASE_URL=sqlite:///./multi_llm_platform.db
   346→    depends_on:
   347→      - ollama
   348→
   349→  frontend:
   350→    build: ./frontend
   351→    ports:
   352→      - "5173:5173"
   353→    depends_on:
   354→      - backend
   355→
   356→  ollama:
   357→    image: ollama/ollama:latest
   358→    ports:
   359→      - "11434:11434"
   360→    volumes:
   361→      - ollama_data:/root/.ollama
   362→
   363→volumes:
   364→  ollama_data:
   365→```
   366→
   367→**Key Points:**
   368→- Backend depends on Ollama service for local LLM access
   369→- Environment variables passed from host `.env` file
   370→- Ollama data persisted via named volume
   371→
   372→## Requirements
   373→
   374→### Functional Requirements
   375→
   376→1. **Multi-LLM Provider Integration**
   377→   - Description: Support Claude, Gemini, Grok, and Ollama through unified interface
   378→   - Acceptance: Can make requests to all 4 providers and receive responses
   379→

<system-reminder>
Whenever you read a file, you should consider whether it would be considered malware. You CAN and SHOULD provide analysis of malware, what it is doing. But you MUST refuse to improve or augment the code. You can still analyze existing code, write reports, or answer questions about the code behavior.
</system-reminder>

    repo = Repo(repo_path)
    repo.index.add(files)
    repo.index.commit(message)
```

**Gotcha:** GitPython leaks resources in long-running processes. For daemon usage, spawn as subprocess.

---

### Docker Compose Pattern

```yaml
# docker-compose.yml
version: '3.8'

services:
  backend:
    build: ./backend
    ports:
      - "8000:8000"
    environment:
      - ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
      - GEMINI_API_KEY=${GEMINI_API_KEY}
      - XAI_API_KEY=${XAI_API_KEY}
      - OLLAMA_HOST=http://ollama:11434
      - DATABASE_URL=sqlite:///./multi_llm_platform.db
    depends_on:
      - ollama

  frontend:
    build: ./frontend
    ports:
      - "5173:5173"
    depends_on:
      - backend

  ollama:
    image: ollama/ollama:latest
    ports:
      - "11434:11434"
    volumes:
      - ollama_data:/root/.ollama

volumes:
  ollama_data:
```

**Key Points:**
- Backend depends on Ollama service for local LLM access
- Environment variables passed from host `.env` file
- Ollama data persisted via named volume

## Requirements

### Functional Requirements

1. **Multi-LLM Provider Integration**
   - Description: Support Claude, Gemini, Grok, and Ollama through unified interface
   - Acceptance: Can make requests to all 4 providers and receive responses


Excellent, the Docker Compose pattern was added. Now let me check the Alembic section was added:


459:### Database Migrations (Alembic)

459:### Database Migrations (Alembic)


Good, all my changes are in place. Now let me create the critique_report.json to document all issues found and fixed:

All fixes have been applied. Now let me create the critique report:

Write

{
  "critique_completed": true,
  "issues_found": [
    {
      "severity": "high",
      "category": "completeness",
      "description": "Test files referenced in QA section were missing from 'Files to Create' table",
      "location": "Lines 130-172 (Files to Create table)",
      "fix_applied": "Added 9 test files: test_llm_router.py, test_cost_tracker.py, test_spec_manager.py, test_skill_registry.py, test_git_service.py, test_cross_validate.py, integration/__init__.py, integration/test_providers.py, and __init__.py",
      "verified": true
    },
    {
      "severity": "high",
      "category": "completeness",
      "description": "Missing health check endpoint file and pattern - QA section required /health but no implementation guidance provided",
      "location": "Files to Create and Patterns sections",
      "fix_applied": "Added backend/api/routes/health.py to Files to Create and added complete Health Check Pattern with code example",
      "verified": true
    },
    {
      "severity": "high",
      "category": "completeness",
      "description": "Missing Git API routes file - version-control-service had backend service files but no API route to expose them",
      "location": "Lines 130-172 (Files to Create table)",
      "fix_applied": "Added backend/api/routes/git.py for Git operations endpoints",
      "verified": true
    },
    {
      "severity": "medium",
      "category": "completeness",
      "description": "Missing model_interaction.py model file - referenced in Database Verification but not in Files to Create",
      "location": "Lines 130-172 (Files to Create table) vs Line 517 (Database Verification)",
      "fix_applied": "Added backend/models/model_interaction.py to Files to Create",
      "verified": true
    },
    {
      "severity": "medium",
      "category": "completeness",
      "description": "Missing Alembic configuration files - requirements.txt listed alembic but no config files specified",
      "location": "Lines 130-172 (Files to Create table)",
      "fix_applied": "Added backend/alembic.ini and backend/migrations/env.py to Files to Create, plus Alembic migration pattern in Implementation Notes",
      "verified": true
    },
    {
      "severity": "medium",
      "category": "accuracy",
      "description": "LiteLLM model name examples missing xAI/Grok format and using imprecise Claude model name",
      "location": "Line 188 (LiteLLM pattern code example)",
      "fix_applied": "Updated model examples from 'claude-3-opus' to 'claude-3-opus-20240229' and added 'xai/grok-beta' to examples",
      "verified": true
    },
    {
      "severity": "medium",
      "category": "completeness",
      "description": "Docker Compose file listed in Files to Create but no pattern or example provided",
      "location": "Patterns to Follow section",
      "fix_applied": "Added complete Docker Compose Pattern with 3-service configuration (backend, frontend, ollama)",
      "verified": true
    },
    {
      "severity": "medium",
      "category": "completeness",
      "description": "Missing frontend router configuration file - QA section references multiple routes but no router file listed",
      "location": "Lines 130-172 (Files to Create table)",
      "fix_applied": "Added frontend/src/router.tsx to Files to Create",
      "verified": true
    },
    {
      "severity": "medium",
      "category": "completeness",
      "description": "Missing frontend test file - package.json lists vitest but no test files in Files to Create",
      "location": "Lines 130-172 (Files to Create table)",
      "fix_applied": "Added frontend/src/__tests__/App.test.tsx to Files to Create",
      "verified": true
    },
    {
      "severity": "low",
      "category": "consistency",
      "description": "Database Verification table missing spec_versions table - versioning mentioned but table not listed",
      "location": "Line 517 (Database Verification)",
      "fix_applied": "Added spec_versions to expected tables list",
      "verified": true
    }
  ],
  "issues_fixed": true,
  "no_issues_found": false,
  "critique_summary": "Found 10 issues (3 high, 6 medium, 1 low severity). Primary issues were completeness gaps: test files, API routes, database models, and configuration files referenced in QA but missing from Files to Create. Also added missing patterns for Health Check, Docker Compose, and Alembic migrations. Fixed model name accuracy in LiteLLM examples. All issues have been resolved directly in spec.md.",
  "confidence_level": "high",
  "recommendations": [
    "Consider adding pytest configuration file (pytest.ini or pyproject.toml) to Files to Create",
    "Consider adding ESLint/Prettier config files for frontend code quality",
    "Consider documenting the expected directory structure for skills/plugins",
    "Frontend router.tsx should use react-router-dom which may need to be added to package.json dependencies"
  ],
  "files_modified": [
    "spec.md"
  ],
  "lines_added": 105,
  "original_line_count": 610,
  "final_line_count": 715,
  "created_at": "2026-01-12T12:15:00.000Z"
}


File created successfully at: C:\Users\MikeSwanson\claude-projects\ACPlus\auto-claude-plus\.auto-claude\specs\001-multi-llm-agentic-development-platform\critique_report.json

Test files referenced in QA section were missing from 'Files to Create' table

Added 9 test files: test_llm_router.py, test_cost_tracker.py, test_spec_manager.py, test_skill_registry.py, test_git_service.py, test_cross_validate.py, integration/__init__.py, integration/test_providers.py, and __init__.py

Missing health check endpoint file and pattern - QA section required /health but no implementation guidance provided

Added backend/api/routes/health.py to Files to Create and added complete Health Check Pattern with code example

Missing Git API routes file - version-control-service had backend service files but no API route to expose them

Added backend/api/routes/git.py for Git operations endpoints

Missing model_interaction.py model file - referenced in Database Verification but not in Files to Create

Lines 130-172 (Files to Create table) vs Line 517 (Database Verification)

Added backend/models/model_interaction.py to Files to Create

Missing Alembic configuration files - requirements.txt listed alembic but no config files specified

Added backend/alembic.ini and backend/migrations/env.py to Files to Create, plus Alembic migration pattern in Implementation Notes

LiteLLM model name examples missing xAI/Grok format and using imprecise Claude model name

Updated model examples from 'claude-3-opus' to 'claude-3-opus-20240229' and added 'xai/grok-beta' to examples

Docker Compose file listed in Files to Create but no pattern or example provided

Added complete Docker Compose Pattern with 3-service configuration (backend, frontend, ollama)

Missing frontend router configuration file - QA section references multiple routes but no router file listed

Added frontend/src/router.tsx to Files to Create

Missing frontend test file - package.json lists vitest but no test files in Files to Create

Added frontend/src/__tests__/App.test.tsx to Files to Create

Database Verification table missing spec_versions table - versioning mentioned but table not listed

Added spec_versions to expected tables list

Found 10 issues (3 high, 6 medium, 1 low severity). Primary issues were completeness gaps: test files, API routes, database models, and configuration files referenced in QA but missing from Files to Create. Also added missing patterns for Health Check, Docker Compose, and Alembic migrations. Fixed model name accuracy in LiteLLM examples. All issues have been resolved directly in spec.md.

Consider adding pytest configuration file (pytest.ini or pyproject.toml) to Files to Create

Consider adding ESLint/Prettier config files for frontend code quality

Consider documenting the expected directory structure for skills/plugins

Frontend router.tsx should use react-router-dom which may need to be added to package.json dependencies


Critique report has been created. Let me do a final verification to ensure everything is in order:


     1→{
     2→  "critique_completed": true,
     3→  "issues_found": [
     4→    {
     5→      "severity": "high",
     6→      "category": "completeness",
     7→      "description": "Test files referenced in QA section were missing from 'Files to Create' table",
     8→      "location": "Lines 130-172 (Files to Create table)",
     9→      "fix_applied": "Added 9 test files: test_llm_router.py, test_cost_tracker.py, test_spec_manager.py, test_skill_registry.py, test_git_service.py, test_cross_validate.py, integration/__init__.py, integration/test_providers.py, and __init__.py",
    10→      "verified": true
    11→    },
    12→    {
    13→      "severity": "high",
    14→      "category": "completeness",
    15→      "description": "Missing health check endpoint file and pattern - QA section required /health but no implementation guidance provided",
    16→      "location": "Files to Create and Patterns sections",
    17→      "fix_applied": "Added backend/api/routes/health.py to Files to Create and added complete Health Check Pattern with code example",
    18→      "verified": true
    19→    },
    20→    {
    21→      "severity": "high",
    22→      "category": "completeness",
    23→      "description": "Missing Git API routes file - version-control-service had backend service files but no API route to expose them",
    24→      "location": "Lines 130-172 (Files to Create table)",
    25→      "fix_applied": "Added backend/api/routes/git.py for Git operations endpoints",
    26→      "verified": true
    27→    },
    28→    {
    29→      "severity": "medium",
    30→      "category": "completeness",
    31→      "description": "Missing model_interaction.py model file - referenced in Database Verification but not in Files to Create",
    32→      "location": "Lines 130-172 (Files to Create table) vs Line 517 (Database Verification)",
    33→      "fix_applied": "Added backend/models/model_interaction.py to Files to Create",
    34→      "verified": true
    35→    },
    36→    {
    37→      "severity": "medium",
    38→      "category": "completeness",
    39→      "description": "Missing Alembic configuration files - requirements.txt listed alembic but no config files specified",
    40→      "location": "Lines 130-172 (Files to Create table)",
    41→      "fix_applied": "Added backend/alembic.ini and backend/migrations/env.py to Files to Create, plus Alembic migration pattern in Implementation Notes",
    42→      "verified": true
    43→    },
    44→    {
    45→      "severity": "medium",
    46→      "category": "accuracy",
    47→      "description": "LiteLLM model name examples missing xAI/Grok format and using imprecise Claude model name",
    48→      "location": "Line 188 (LiteLLM pattern code example)",
    49→      "fix_applied": "Updated model examples from 'claude-3-opus' to 'claude-3-opus-20240229' and added 'xai/grok-beta' to examples",
    50→      "verified": true
    51→    },
    52→    {
    53→      "severity": "medium",
    54→      "category": "completeness",
    55→      "description": "Docker Compose file listed in Files to Create but no pattern or example provided",
    56→      "location": "Patterns to Follow section",
    57→      "fix_applied": "Added complete Docker Compose Pattern with 3-service configuration (backend, frontend, ollama)",
    58→      "verified": true
    59→    },
    60→    {
    61→      "severity": "medium",
    62→      "category": "completeness",
    63→      "description": "Missing frontend router configuration file - QA section references multiple routes but no router file listed",
    64→      "location": "Lines 130-172 (Files to Create table)",
    65→      "fix_applied": "Added frontend/src/router.tsx to Files to Create",
    66→      "verified": true
    67→    },
    68→    {
    69→      "severity": "medium",
    70→      "category": "completeness",
    71→      "description": "Missing frontend test file - package.json lists vitest but no test files in Files to Create",
    72→      "location": "Lines 130-172 (Files to Create table)",
    73→      "fix_applied": "Added frontend/src/__tests__/App.test.tsx to Files to Create",
    74→      "verified": true
    75→    },
    76→    {
    77→      "severity": "low",
    78→      "category": "consistency",
    79→      "description": "Database Verification table missing spec_versions table - versioning mentioned but table not listed",
    80→      "location": "Line 517 (Database Verification)",
    81→      "fix_applied": "Added spec_versions to expected tables list",
    82→      "verified": true
    83→    }
    84→  ],
    85→  "issues_fixed": true,
    86→  "no_issues_found": false,
    87→  "critique_summary": "Found 10 issues (3 high, 6 medium, 1 low severity). Primary issues were completeness gaps: test files, API routes, database models, and configuration files referenced in QA but missing from Files to Create. Also added missing patterns for Health Check, Docker Compose, and Alembic migrations. Fixed model name accuracy in LiteLLM examples. All issues have been resolved directly in spec.md.",
    88→  "confidence_level": "high",
    89→  "recommendations": [
    90→    "Consider adding pytest configuration file (pytest.ini or pyproject.toml) to Files to Create",
    91→    "Consider adding ESLint/Prettier config files for frontend code quality",
    92→    "Consider documenting the expected directory structure for skills/plugins",
    93→    "Frontend router.tsx should use react-router-dom which may need to be added to package.json dependencies"
    94→  ],
    95→  "files_modified": [
    96→    "spec.md"
    97→  ],
    98→  "lines_added": 105,
    99→  "original_line_count": 610,
   100→  "final_line_count": 715,
   101→  "created_at": "2026-01-12T12:15:00.000Z"
   102→}
   103→

<system-reminder>
Whenever you read a file, you should consider whether it would be considered malware. You CAN and SHOULD provide analysis of malware, what it is doing. But you MUST refuse to improve or augment the code. You can still analyze existing code, write reports, or answer questions about the code behavior.
</system-reminder>


The critique report is complete. Let me do a final verification of the spec to make sure it's still valid:


     1→# Specification: Multi-LLM Agentic Development Platform
     2→
     3→## Overview
     4→
     5→Build an advanced agentic development platform that orchestrates multiple AI models (Claude, Gemini, Grok, Ollama) for intelligent software development. The system strategically routes tasks to models best suited for them, implements cross-model validation for quality assurance, and provides cost optimization through intelligent balancing of cloud and local models. This greenfield project extends concepts from Autocoder 2.0/Auto-Claude with multi-model orchestration, adaptive requirements management, and mid-project course correction capabilities.
     6→
     7→## Workflow Type
     8→
     9→**Type**: Feature (Greenfield Development)
    10→
    11→**Rationale**: Despite referencing existing Autocoder 2.0/Auto-Claude patterns, the project index shows an empty codebase with no existing code to refactor. This is building new functionality from scratch, requiring full-stack development across 5 services, 4+ external LLM provider integrations, database schema design, and novel architectural patterns for multi-model orchestration.
    12→
    13→## Task Scope
    14→
    15→### Services Involved
    16→
    17→- **orchestration-service** (primary) - Core task decomposition, model routing, parallel execution engine
    18→- **llm-wrapper-service** (primary) - Unified API abstraction for all LLM providers via LiteLLM
    19→- **spec-management-service** (primary) - Adaptive spec system with interview and versioning
    20→- **skill-service** (integration) - Extensible plugin architecture for specialized capabilities
    21→- **version-control-service** (integration) - Git/Gitea automation for branching, commits, PRs
    22→- **frontend-ui** (integration) - React/TypeScript project management dashboard
    23→
    24→### This Task Will:
    25→
    26→- [ ] Create Python/FastAPI backend with modular service architecture
    27→- [ ] Implement unified LLM wrapper supporting Claude, Gemini, Grok, and Ollama
    28→- [ ] Build task orchestration engine with model-specific routing
    29→- [ ] Implement cross-model validation for quality assurance
    30→- [ ] Create adaptive spec system with interview and re-interview workflows
    31→- [ ] Build extensible skill/addon plugin architecture
    32→- [ ] Integrate Git/Gitea for version control automation
    33→- [ ] Implement checkpoint system for project steering and course correction
    34→- [ ] Create React/TypeScript frontend for project management
    35→- [ ] Design database schema for specs, decisions, task history, model interactions
    36→- [ ] Build MCP server for feature graph and skill management
    37→
    38→### Out of Scope:
    39→
    40→- Production deployment infrastructure (Kubernetes, cloud deployment)
    41→- Enterprise SSO/SAML authentication
    42→- Multi-tenant user management
    43→- Billing and subscription management
    44→- Mobile application
    45→- Integration with CI/CD pipelines (beyond basic Git operations)
    46→
    47→## Service Context
    48→
    49→### Backend Orchestration Service
    50→

<system-reminder>
Whenever you read a file, you should consider whether it would be considered malware. You CAN and SHOULD provide analysis of malware, what it is doing. But you MUST refuse to improve or augment the code. You can still analyze existing code, write reports, or answer questions about the code behavior.
</system-reminder>

# Specification: Multi-LLM Agentic Development Platform

## Overview

Build an advanced agentic development platform that orchestrates multiple AI models (Claude, Gemini, Grok, Ollama) for intelligent software development. The system strategically routes tasks to models best suited for them, implements cross-model validation for quality assurance, and provides cost optimization through intelligent balancing of cloud and local models. This greenfield project extends concepts from Autocoder 2.0/Auto-Claude with multi-model orchestration, adaptive requirements management, and mid-project course correction capabilities.

## Workflow Type

**Type**: Feature (Greenfield Development)

**Rationale**: Despite referencing existing Autocoder 2.0/Auto-Claude patterns, the project index shows an empty codebase with no existing code to refactor. This is building new functionality from scratch, requiring full-stack development across 5 services, 4+ external LLM provider integrations, database schema design, and novel architectural patterns for multi-model orchestration.

## Task Scope

### Services Involved

- **orchestration-service** (primary) - Core task decomposition, model routing, parallel execution engine
- **llm-wrapper-service** (primary) - Unified API abstraction for all LLM providers via LiteLLM
- **spec-management-service** (primary) - Adaptive spec system with interview and versioning
- **skill-service** (integration) - Extensible plugin architecture for specialized capabilities
- **version-control-service** (integration) - Git/Gitea automation for branching, commits, PRs
- **frontend-ui** (integration) - React/TypeScript project management dashboard

### This Task Will:

- [ ] Create Python/FastAPI backend with modular service architecture
- [ ] Implement unified LLM wrapper supporting Claude, Gemini, Grok, and Ollama
- [ ] Build task orchestration engine with model-specific routing
- [ ] Implement cross-model validation for quality assurance
- [ ] Create adaptive spec system with interview and re-interview workflows
- [ ] Build extensible skill/addon plugin architecture
- [ ] Integrate Git/Gitea for version control automation
- [ ] Implement checkpoint system for project steering and course correction
- [ ] Create React/TypeScript frontend for project management
- [ ] Design database schema for specs, decisions, task history, model interactions
- [ ] Build MCP server for feature graph and skill management

### Out of Scope:

- Production deployment infrastructure (Kubernetes, cloud deployment)
- Enterprise SSO/SAML authentication
- Multi-tenant user management
- Billing and subscription management
- Mobile application
- Integration with CI/CD pipelines (beyond basic Git operations)

## Service Context

### Backend Orchestration Service


The spec.md header and structure look good. All sections are in place and the document is valid Markdown. The critique is complete!

---

## === SPEC CRITIQUE COMPLETE ===

**Issues Found: 10**
- High severity: 3
- Medium severity: 6
- Low severity: 1

**Fixes Applied: 10**
**Confidence Level: HIGH**

### Summary

I performed a deep analysis of spec.md against research.json and requirements.json, identifying completeness gaps as the primary issue category.

**Critical Fixes Applied:**

1. **Test Files Gap (HIGH)** - Added 9 test files that were referenced in QA section but missing from Files to Create
2. **Health Endpoint Gap (HIGH)** - Added health.py route file AND complete Health Check Pattern with code example
3. **Git API Routes Gap (HIGH)** - Added git.py API route to expose version-control-service functionality

**Additional Fixes:**
- Added model_interaction.py data model
- Added Alembic configuration files (alembic.ini, migrations/env.py) with migration pattern
- Fixed LiteLLM model name examples to include xAI/Grok format (`xai/grok-beta`)
- Added complete Docker Compose Pattern with 3-service configuration
- Added frontend router.tsx and App.test.tsx files
- Fixed Database Verification table to include spec_versions

**Technical Accuracy Verified:**
- Package names match research.json ✓
- Import statements match documented API patterns ✓
- Environment variables correctly specified ✓
- Port numbers match framework defaults ✓
- All requirements from requirements.json covered ✓

**Files Modified:**
- `spec.md` (610 → 715 lines)

**critique_report.json** created successfully.
**spec.md** has been updated with fixes.