Excellent! Now I have a comprehensive understanding of the GuruRMM server architecture. Let me compile my findings into a detailed report.

## GuruRMM Server Architecture Analysis - Policy Delivery & Alert System Design

Based on my thorough exploration of the codebase, here's a detailed report on the server architecture:

---

### 1. API ROUTE PATTERNS AND EXTENSIBILITY

**Current Route Structure** (from `/C:\Users\MikeSwanson\claude-projects\gururmm\server\src\api\mod.rs`):

The API is built with **Axum**, a Rust web framework, using a modular routing pattern:

```rust
Router::new()
    // Authentication (POST/GET)
    .route("/auth/login", post(auth::login))
    .route("/auth/register", post(auth::register))
    .route("/auth/me", get(auth::me))

    // Clients & Sites (full CRUD)
    .route("/clients", get/post/put/delete)
    .route("/sites", get/post/put/delete)

    // Agents (management)
    .route("/agents", get/post)
    .route("/agents/:id", get/delete)
    .route("/agents/:id/move", post)
    .route("/agents/:id/update", post)
    .route("/agents/:id/state", get)

    // Metrics (time-series retrieval)
    .route("/agents/:id/metrics", get)
    .route("/metrics/summary", get)

    // Commands (dispatch & history)
    .route("/agents/:id/command", post)
    .route("/commands", get)
    .route("/commands/:id", get)
```

**How to Add New Routes:**
The pattern is straightforward. New routes would be added as:
1. Create a new module file in `server/src/api/` (e.g., `policies.rs`)
2. Add handler functions with signatures like: `async fn handler(State(state): State<AppState>, auth: AuthUser, ...) -> Result<Json<T>, (StatusCode, String)>`
3. Register in `api/mod.rs` via `.route("/policies/...", post/get/put/delete(module::handler))`

**Key Advantages for Policy Routes:**
- Type-safe error handling with `StatusCode` responses
- JWT authentication via `AuthUser` extractor (automatic on all routes)
- State injection provides access to: DB pool, WebSocket connections, config, update manager
- Supports path parameters: `/sites/:site_id/policies`
- Query parameters supported for filtering: `/agents?site_id=...&status=...`

---

### 2. WEBSOCKET MESSAGE FLOW TO AGENTS

**Architecture** (from `/C:\Users\MikeSwanson\claude-projects\gururmm\server\src\ws\mod.rs`):

The server maintains **real-time bidirectional communication** via WebSocket with a clean message envelope system:

**Agent → Server Message Types:**
```rust
pub enum AgentMessage {
    Auth(AuthPayload),           // Initial handshake
    Metrics(MetricsPayload),      // CPU, memory, disk, temps (every ~30s)
    NetworkState(NetworkStatePayload),  // Interface list
    CommandResult(CommandResultPayload),  // Command execution results
    WatchdogEvent(WatchdogEventPayload),  // Service/process alerts
    UpdateResult(UpdateResultPayload),    // Agent update completion
    Heartbeat,                   // Keep-alive
}
```

**Server → Agent Message Types:**
```rust
pub enum ServerMessage {
    AuthAck(AuthAckPayload),     // Authentication response
    Command(CommandPayload),      // Execute shell/PowerShell/Python
    ConfigUpdate(serde_json::Value),  // FLEXIBLE JSON config (unused currently!)
    Update(UpdatePayload),        // Push binary update
    Ack { message_id: Option<String> },  // Acknowledgment
    Error { code: String, message: String },  // Error notification
}
```

**Message Flow Diagram:**

```
Agent connects → sends Auth → server verifies & stores in AgentConnections map

While connected:
  - Agent sends Metrics → stored in DB metrics table (time-series)
  - Agent sends WatchdogEvent → logged (not stored systematically yet)
  - Server sends Command → forwarded via channel → agent executes
  - Agent sends CommandResult → stored in commands table with exit code/output
  - Server can send ConfigUpdate → NOT YET IMPLEMENTED (empty JSON)
  - Server can send Update → agent downloads & installs new binary

Agent disconnects → removed from AgentConnections, status set to "offline"
```

**Critical for Policy System:**
- `ConfigUpdate(serde_json::Value)` message type **already exists** but is not used
- This is the perfect hook for delivering policies to agents
- Uses JSON serialization: flexible for different policy schemas
- Delivery mechanism: immediate via `agents.send_to(&agent_id, msg).await`

**Connection Management:**
```rust
pub struct AgentConnections {
    connections: HashMap<Uuid, mpsc::Sender<ServerMessage>>,
}

// Send to specific agent
pub async fn send_to(&self, agent_id: &Uuid, msg: ServerMessage) -> bool {
    // Returns false if agent not connected (offline)
}

// Check connection status
pub fn is_connected(&agent_id: &Uuid) -> bool
```

---

### 3. CONFIGURATION/COMMAND DELIVERY MECHANISM

**Current Command System** (from `server/src/api/commands.rs`):

Commands are sent via REST API and dispatched immediately to connected agents:

```rust
POST /api/agents/:id/command
{
    "command_type": "shell",      // or "powershell", "python", "script"
    "command": "systemctl status nginx",
    "timeout_seconds": 300,
    "elevated": false
}

Response:
{
    "command_id": "uuid",
    "status": "running|pending",
    "message": "Command sent to agent | Agent offline, queued"
}
```

**Execution Flow:**
1. REST endpoint creates command record in DB (status = "pending")
2. Checks if agent is connected: `agents.is_connected(&agent_id)`
3. If connected: sends via WebSocket immediately (status = "running")
4. If offline: command stays "pending" until agent reconnects
5. Agent executes and returns `CommandResult`
6. Server updates DB with exit code, stdout, stderr

**For Policies, This Suggests:**

Policies can follow the same pattern:
1. Create policy in DB (new `policies` table)
2. Send via `ConfigUpdate` message when agent connects
3. Store delivery acknowledgment in DB
4. For offline agents: queue policies, deliver on reconnection
5. Track policy compliance via metrics from agents

---

### 4. EXISTING ALERTING & THRESHOLD PATTERNS

**What Currently Exists:**

**Metrics Storage** (from `server/src/db/metrics.rs`):
- Every agent sends metrics every ~30s (timestamp-based)
- Stored in PostgreSQL `metrics` table with time-series indexing
- Fields available: CPU%, memory%, disk%, network I/O, uptime, logged-in user, IPs, temperatures

```rust
pub struct Metrics {
    pub cpu_percent: Option<f32>,
    pub memory_percent: Option<f32>,
    pub memory_used_bytes: Option<i64>,
    pub disk_percent: Option<f32>,
    pub cpu_temp_celsius: Option<f32>,
    pub gpu_temp_celsius: Option<f32>,
    pub temperatures: Option<serde_json::Value>,  // Array of sensors
}
```

**Summary Queries Available:**
- `get_metrics_summary()` → aggregates last hour: AVG(cpu), AVG(memory), SUM(network)
- `get_agent_metrics_range(start, end)` → time-range queries
- Index on `(agent_id, timestamp DESC)` for efficient historical queries

**Watchdog Events** (from `ws/mod.rs`):
```rust
AgentMessage::WatchdogEvent(WatchdogEventPayload) {
    pub name: String,           // "nginx", "mysql", etc.
    pub event: String,          // "stopped", "restarted", etc.
    pub details: Option<String>,
}
```
- Currently **logged but not stored in DB** (comment: "table exists but we'll add the insert function later")
- Perfect for alert triggering

**What's NOT Yet Implemented:**
- No `alert_thresholds` table
- No `alert_policies` table
- No alert evaluation engine
- No notification mechanism (email, Slack, etc.)
- No alert history/acknowledgment tracking

---

### 5. DATABASE SCHEMA FOR POLICY SYSTEM

**Current Tables** (from migrations):

```
agents
├── id, hostname, status, os_type, version, last_seen
├── device_id (hardware-derived ID)
└── site_id (FK to sites)

sites
├── id, name, site_code, client_id
└── api_key_hash (shared by all agents at site)

clients
└── id, name, code, is_active

metrics (time-series)
├── agent_id, timestamp
├── cpu_percent, memory_percent, disk_percent
├── network_rx_bytes, network_tx_bytes
├── uptime_seconds, logged_in_user, public_ip
└── cpu_temp_celsius, gpu_temp_celsius, temperatures JSONB

commands
├── agent_id, command_type, status
├── exit_code, stdout, stderr
└── created_by (user who issued it)

watchdog_events
├── agent_id, timestamp, service_name
└── event_type, details

agent_state (latest snapshot, not time-series)
├── agent_id
├── network_interfaces JSONB
├── uptime_seconds, logged_in_user, public_ip
└── metrics_updated_at, network_updated_at
```

**Schema Needed for Policies:**

```sql
-- Policy definitions (templates)
policies
├── id, site_id, name, description
├── type ('cpu_threshold' | 'memory_threshold' | 'disk_threshold' | 'service_monitoring' | 'custom')
├── enabled BOOLEAN, priority ('low' | 'medium' | 'high' | 'critical')
└── created_at, updated_at

-- Policy conditions/thresholds
policy_conditions
├── id, policy_id
├── metric ('cpu_percent' | 'memory_percent' | 'disk_percent' | 'temperature' | custom)
├── operator ('>' | '<' | '==' | '!=' for numeric/string comparisons)
├── threshold_value, duration_seconds (how long before triggering)
└── created_at

-- Alert rules (what happens when policy triggered)
alert_rules
├── id, policy_id
├── action_type ('notify' | 'command' | 'escalate')
├── action_config JSONB (email addresses, command to run, escalation level)
└── enabled BOOLEAN

-- Alert history (for dashboards, auditing)
alerts
├── id, policy_id, agent_id, timestamp
├── status ('triggered' | 'resolved' | 'acknowledged')
├── value (actual metric value at trigger)
├── acknowledged_by, acknowledged_at
└── resolved_at
```

---

### 6. ARCHITECTURE FOR NEW POLICY FEATURES

**Proposed Implementation Path:**

**1. REST API Layer** (`server/src/api/policies.rs`):
```
POST   /api/policies                    → Create policy
GET    /api/policies                    → List policies
GET    /api/policies/:id                → Get policy details
PUT    /api/policies/:id                → Update policy
DELETE /api/policies/:id                → Delete policy

POST   /api/policies/:id/deploy         → Push to agents
GET    /api/policies/:id/deployment     → Check rollout status
GET    /api/alerts                      → List alerts (with filtering)
POST   /api/alerts/:id/acknowledge      → Mark alert as reviewed
```

**2. Database Module** (`server/src/db/policies.rs`):
- CRUD operations for policies, conditions, rules
- Batch queries: "all active policies for site X"
- Alert insertion, retrieval, acknowledgment

**3. Evaluation Engine** (`server/src/policy_engine.rs`):
- Background task that runs every N seconds
- Queries latest metrics from `metrics` table
- Evaluates each active policy's conditions
- Creates alerts in DB when conditions met
- Handles escalation (repeat alerts, notify admins)

**4. WebSocket Integration** (`server/src/ws/mod.rs`):
- When policy is deployed: send `ConfigUpdate` with policy JSON
- Agent periodically evaluates policies locally
- Agent sends back `WatchdogEvent` for policy violations
- Server correlation: match agent-reported events with expected policies

---

### 7. KEY ARCHITECTURAL ADVANTAGES

1. **Real-time Delivery**: WebSocket channel already exists for instant config pushes
2. **Metrics Foundation**: Metrics stored with agent ID + timestamp, perfect for threshold evaluation
3. **Existing Audit Trail**: Commands already track who initiated them → policies can do same
4. **Site-based Scoping**: Sites table provides organizational boundaries for policy templates
5. **JSON Flexibility**: `ConfigUpdate(serde_json::Value)` allows any policy schema
6. **Offline Handling**: Command system already queues for offline agents → model for policies
7. **Modular**: New modules don't require changes to core WebSocket or agent authentication

---

### 8. FILE PATHS FOR REFERENCE

- **API Routes**: `/C:\Users\MikeSwanson\claude-projects\gururmm\server\src\api\mod.rs`
- **WebSocket Core**: `/C:\Users\MikeSwanson\claude-projects\gururmm\server\src\ws\mod.rs`
- **Message Payloads**: Lines 77-251 in ws/mod.rs (all message types defined)
- **Database Schema**: `/C:\Users\MikeSwanson\claude-projects\gururmm\server\migrations\` (all 5 migration files)
- **Commands Handler**: `/C:\Users\MikeSwanson\claude-projects\gururmm\server\src\api\commands.rs`
- **Metrics Handler**: `/C:\Users\MikeSwanson\claude-projects\gururmm\server\src\api\metrics.rs`
- **Metrics DB Queries**: `/C:\Users\MikeSwanson\claude-projects\gururmm\server\src\db\metrics.rs`

---

### Summary

The GuruRMM server is **well-architected for policy delivery**. The ConfigUpdate message type exists but is currently unused—this is your primary vehicle for policy distribution. The metrics storage is solid with proper time-series indexing, the command dispatch pattern provides a proven model for off-line queueing, and the site-based hierarchy enables policy templates. The main work is:

1. Create policy/threshold schema tables
2. Implement REST API for policy CRUD and deployment
3. Build evaluation engine (background task or triggered on metrics ingestion)
4. Wire up alert history and escalation rules

agentId: a0cdffc (for resuming to continue this agent's work if needed)

Prompt:
Explore the GuruRMM server to understand the API and WebSocket architecture. Look at:

1. server/src/api/mod.rs - API route structure
2. server/src/ws/mod.rs - WebSocket message handling
3. How server sends configuration/commands to agents
4. Any existing alert or notification patterns

I need to understand the server architecture to design policy delivery and alert threshold evaluation. Report:
- API route patterns and how new routes would be added
- WebSocket message flow to agents
- Any existing alerting or threshold checking logic


I now have comprehensive information about the codebase. Let me summarize the key findings and then design the policy system. Before writing the plan, I should clarify a few things with the user about their specific requirements.

Key findings:
1. **Database**: Has a good hierarchy (Client → Site → Agent), JSONB support for flexible data
2. **Agent**: Already has ConfigUpdate message type (unused), TrayPolicy as existing policy example
3. **Server**: WebSocket delivery ready, metrics stored with timestamps for threshold evaluation

I need to ask the user some clarifying questions about:
1. Policy inheritance model - strict override vs merge
2. Alert notification methods (email, webhook, dashboard only?)
3. Priority for implementation phases

Let me ask these questions before finalizing the plan.


Excellent exploration results. I have a clear picture of the architecture. Before finalizing the plan, I have a few clarifying questions:


AskUserQuestion

Question: How should conflicting policy settings be resolved when multiple policies apply to the same agent?

Choices:
Later policies in the stack completely override earlier ones for the same setting
When conflicts occur, apply the more restrictive/secure value
Each policy has a priority number; highest priority wins conflicts

Question: What alert notification methods do you need initially?

Choices:
Alerts visible in dashboard with badge counts - simplest to implement first
Dashboard alerts plus email notifications to configured addresses
Dashboard alerts plus webhook calls to external systems (Slack, Teams, etc.)

Question: Should policies be assignable at which levels?

Choices:
Full hierarchy: base policy → client → site → individual agent overrides
No global default, start at client level
Simpler model, policies only at site and agent level


User has answered your questions: "How should conflicting policy settings be resolved when multiple policies apply to the same agent?"="Last wins (stack order)", "What alert notification methods do you need initially?"="Dashboard only (Recommended)", "Should policies be assignable at which levels?"="Client + Site + Agent". You can now continue with the user's answers in mind.

Question: How should conflicting policy settings be resolved when multiple policies apply to the same agent?

Choices:
Later policies in the stack completely override earlier ones for the same setting
When conflicts occur, apply the more restrictive/secure value
Each policy has a priority number; highest priority wins conflicts

Question: What alert notification methods do you need initially?

Choices:
Alerts visible in dashboard with badge counts - simplest to implement first
Dashboard alerts plus email notifications to configured addresses
Dashboard alerts plus webhook calls to external systems (Slack, Teams, etc.)

Question: Should policies be assignable at which levels?

Choices:
Full hierarchy: base policy → client → site → individual agent overrides
No global default, start at client level
Simpler model, policies only at site and agent level


The user has answered:
1. **Merge Logic**: Last wins (stack order) - later policies override earlier ones
2. **Notifications**: Dashboard only (start simple)
3. **Assignment levels**: Client + Site + Agent (no global base policy)

Now I have all the information I need. Let me write the plan to the plan file. First, let me read the existing plan file to see what's there (it was for SSO), then overwrite it with the new policy system plan.


Now I have all the requirements. Let me design the implementation plan:


Task

Design policy system architecture

Prompt:
Design a stackable policy system for GuruRMM based on these requirements and codebase findings:

**Requirements:**
- Policies assignable at: Client, Site, and Agent levels
- Stack order: Client → Site → Agent (last wins for conflicts)
- Settings include: metrics collection (temps, intervals), alert thresholds (CPU > X% for Y minutes), system tray options, auto-update settings
- Alerts: Dashboard-only initially (no email/webhook)
- Delivery: Via existing ConfigUpdate WebSocket message

**Codebase Context:**
- Database: PostgreSQL with existing tables: clients, sites, agents, metrics, agent_state
- Server: Axum web framework, WebSocket in ws/mod.rs, ServerMessage::ConfigUpdate exists but unused
- Agent: Has ConfigUpdatePayload structure, TrayPolicy already shows policy pattern
- Agent config includes: MetricsConfig (interval, collect_*), WatchdogConfig (services, processes)

**Key Files:**
- server/src/db/ - Database modules (agents.rs, clients.rs, sites.rs, metrics.rs)
- server/src/api/mod.rs - API routes
- server/src/ws/mod.rs - WebSocket handling, ServerMessage enum
- agent/src/config.rs - Agent configuration
- agent/src/transport/mod.rs - Message types including ConfigUpdatePayload

**Design Requirements:**
1. Database schema for policies, policy_assignments, alerts
2. Policy structure that covers: metrics settings, thresholds, tray settings, update settings
3. Server-side policy merge logic (evaluate effective policy for an agent)
4. API endpoints for CRUD and deployment
5. Alert evaluation approach (when to check thresholds)
6. Agent-side policy application

Provide a detailed implementation plan with specific structs, tables, and code patterns.


<bash-notification>
<shell-id>b3b75e2</shell-id>
<output-file>C:\Users\MIKESW~1\AppData\Local\Temp\claude\C--Users-MikeSwanson-claude-projects\tasks\b3b75e2.output</output-file>
<status>failed</status>
<summary>Background command "Deploy server binary" failed with exit code 137.</summary>
Read the output file to retrieve the output.
</bash-notification>


<bash-notification>
<shell-id>bbf3def</shell-id>
<output-file>C:\Users\MIKESW~1\AppData\Local\Temp\claude\C--Users-MikeSwanson-claude-projects\tasks\bbf3def.output</output-file>
<status>failed</status>
<summary>Background command "Deploy updated server with JWT auth" failed with exit code 137.</summary>
Read the output file to retrieve the output.
</bash-notification>


<bash-notification>
<shell-id>b332ee1</shell-id>
<output-file>C:\Users\MIKESW~1\AppData\Local\Temp\claude\C--Users-MikeSwanson-claude-projects\tasks\b332ee1.output</output-file>
<status>failed</status>
<summary>Background command "Check if GuruRMM Docker runs on Jupiter" failed with exit code 137.</summary>
Read the output file to retrieve the output.
</bash-notification>


Now I have a complete picture of the codebase. Let me provide a comprehensive implementation plan for the stackable policy system.

---

# Stackable Policy System for GuruRMM - Implementation Plan

## Executive Summary

This plan designs a hierarchical policy system where policies can be assigned at Client, Site, and Agent levels, with a "last wins" merge strategy (Agent overrides Site overrides Client). The system will use the existing WebSocket `ConfigUpdate` message for policy delivery and evaluate thresholds on the server during metrics ingestion.

---

## 1. Database Schema

### 1.1 New Tables

#### `policies` Table
Stores policy definitions as JSON documents. Each policy is a reusable template that can be assigned to multiple targets.

```sql
-- Policy definitions (reusable policy templates)
CREATE TABLE policies (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    name VARCHAR(255) NOT NULL,
    description TEXT,
    -- Policy content stored as JSONB for flexibility
    policy_data JSONB NOT NULL DEFAULT '{}',
    is_default BOOLEAN DEFAULT FALSE,  -- Mark as system default
    created_at TIMESTAMPTZ DEFAULT NOW(),
    updated_at TIMESTAMPTZ DEFAULT NOW(),
    created_by UUID REFERENCES users(id) ON DELETE SET NULL
);

CREATE INDEX idx_policies_name ON policies(name);
CREATE INDEX idx_policies_default ON policies(is_default) WHERE is_default = TRUE;

-- Trigger for updated_at
CREATE TRIGGER update_policies_updated_at
    BEFORE UPDATE ON policies
    FOR EACH ROW
    EXECUTE FUNCTION update_updated_at_column();
```

#### `policy_assignments` Table
Links policies to targets (clients, sites, or agents) with a priority system.

```sql
-- Policy assignments (links policies to targets)
CREATE TABLE policy_assignments (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    policy_id UUID NOT NULL REFERENCES policies(id) ON DELETE CASCADE,
    -- Target: exactly one of these should be set
    client_id UUID REFERENCES clients(id) ON DELETE CASCADE,
    site_id UUID REFERENCES sites(id) ON DELETE CASCADE,
    agent_id UUID REFERENCES agents(id) ON DELETE CASCADE,
    -- Priority within same level (higher = takes precedence)
    priority INTEGER DEFAULT 0,
    created_at TIMESTAMPTZ DEFAULT NOW(),
    created_by UUID REFERENCES users(id) ON DELETE SET NULL,

    -- Ensure exactly one target is set
    CONSTRAINT policy_assignment_target_check CHECK (
        (client_id IS NOT NULL)::int +
        (site_id IS NOT NULL)::int +
        (agent_id IS NOT NULL)::int = 1
    )
);

CREATE INDEX idx_policy_assignments_policy ON policy_assignments(policy_id);
CREATE INDEX idx_policy_assignments_client ON policy_assignments(client_id) WHERE client_id IS NOT NULL;
CREATE INDEX idx_policy_assignments_site ON policy_assignments(site_id) WHERE site_id IS NOT NULL;
CREATE INDEX idx_policy_assignments_agent ON policy_assignments(agent_id) WHERE agent_id IS NOT NULL;
```

#### `alerts` Table
Stores triggered alerts for dashboard display.

```sql
-- Alerts triggered by threshold evaluation
CREATE TABLE alerts (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    agent_id UUID NOT NULL REFERENCES agents(id) ON DELETE CASCADE,
    -- Alert categorization
    alert_type VARCHAR(50) NOT NULL,  -- 'cpu', 'memory', 'disk', 'temperature', 'service', 'custom'
    severity VARCHAR(20) NOT NULL DEFAULT 'warning',  -- 'info', 'warning', 'critical'
    -- Alert content
    title VARCHAR(255) NOT NULL,
    message TEXT,
    metric_value REAL,  -- The value that triggered the alert
    threshold_value REAL,  -- The threshold that was exceeded
    -- State management
    status VARCHAR(20) DEFAULT 'active',  -- 'active', 'acknowledged', 'resolved'
    acknowledged_at TIMESTAMPTZ,
    acknowledged_by UUID REFERENCES users(id) ON DELETE SET NULL,
    resolved_at TIMESTAMPTZ,
    -- Deduplication: hash of agent_id + alert_type + relevant identifiers
    dedup_key VARCHAR(255),
    -- Timestamps
    triggered_at TIMESTAMPTZ DEFAULT NOW(),
    last_seen_at TIMESTAMPTZ DEFAULT NOW(),
    created_at TIMESTAMPTZ DEFAULT NOW()
);

CREATE INDEX idx_alerts_agent ON alerts(agent_id);
CREATE INDEX idx_alerts_status ON alerts(status);
CREATE INDEX idx_alerts_type_status ON alerts(alert_type, status);
CREATE INDEX idx_alerts_triggered ON alerts(triggered_at DESC);
CREATE INDEX idx_alerts_dedup ON alerts(dedup_key) WHERE status = 'active';
```

#### `alert_thresholds_state` Table
Tracks threshold violation state for "X% for Y minutes" logic.

```sql
-- Threshold violation state tracking (for duration-based alerts)
CREATE TABLE alert_threshold_state (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    agent_id UUID NOT NULL REFERENCES agents(id) ON DELETE CASCADE,
    threshold_key VARCHAR(100) NOT NULL,  -- e.g., 'cpu_warning', 'memory_critical'
    -- State tracking
    violation_started_at TIMESTAMPTZ,
    consecutive_violations INTEGER DEFAULT 0,
    last_value REAL,
    last_checked_at TIMESTAMPTZ DEFAULT NOW(),

    UNIQUE(agent_id, threshold_key)
);

CREATE INDEX idx_threshold_state_agent ON alert_threshold_state(agent_id);
```

---

## 2. Policy Data Structure

### 2.1 Rust Structs (Server-side)

```rust
// server/src/db/policies.rs

use serde::{Deserialize, Serialize};
use sqlx::types::Json;
use uuid::Uuid;

/// Complete policy structure stored as JSONB
#[derive(Debug, Clone, Serialize, Deserialize, Default)]
pub struct PolicyData {
    /// Metrics collection settings
    #[serde(default)]
    pub metrics: Option<MetricsPolicy>,

    /// Alert thresholds
    #[serde(default)]
    pub thresholds: Option<ThresholdsPolicy>,

    /// System tray settings
    #[serde(default)]
    pub tray: Option<TrayPolicy>,

    /// Auto-update settings
    #[serde(default)]
    pub updates: Option<UpdatesPolicy>,

    /// Watchdog configuration
    #[serde(default)]
    pub watchdog: Option<WatchdogPolicy>,
}

/// Metrics collection policy
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct MetricsPolicy {
    /// Collection interval in seconds (10-3600)
    pub interval_seconds: Option<u64>,
    /// Which metrics to collect
    pub collect_cpu: Option<bool>,
    pub collect_memory: Option<bool>,
    pub collect_disk: Option<bool>,
    pub collect_network: Option<bool>,
    pub collect_temperatures: Option<bool>,
    /// Extended metrics
    pub collect_user_info: Option<bool>,
    pub collect_public_ip: Option<bool>,
}

/// Alert thresholds policy
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ThresholdsPolicy {
    /// CPU thresholds
    pub cpu: Option<ResourceThreshold>,
    /// Memory thresholds
    pub memory: Option<ResourceThreshold>,
    /// Disk thresholds
    pub disk: Option<ResourceThreshold>,
    /// Temperature thresholds (Celsius)
    pub cpu_temp: Option<TemperatureThreshold>,
    pub gpu_temp: Option<TemperatureThreshold>,
}

/// Threshold for a resource (CPU/Memory/Disk)
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ResourceThreshold {
    /// Warning threshold percentage
    pub warning_percent: Option<f32>,
    /// Critical threshold percentage
    pub critical_percent: Option<f32>,
    /// Duration in seconds before triggering (0 = immediate)
    pub duration_seconds: Option<u64>,
}

/// Temperature threshold
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct TemperatureThreshold {
    pub warning_celsius: Option<f32>,
    pub critical_celsius: Option<f32>,
}

/// Tray application policy (matches existing TrayPolicy)
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct TrayPolicy {
    pub enabled: Option<bool>,
    pub show_status: Option<bool>,
    pub show_info: Option<bool>,
    pub allow_force_checkin: Option<bool>,
    pub allow_view_logs: Option<bool>,
    pub allow_open_dashboard: Option<bool>,
    pub allow_stop_agent: Option<bool>,
    pub dashboard_url: Option<String>,
    pub tooltip_text: Option<String>,
    pub custom_icon: Option<String>,
}

/// Auto-update policy
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct UpdatesPolicy {
    /// Enable automatic updates
    pub auto_update: Option<bool>,
    /// Maintenance window (24h format, e.g., "02:00-06:00")
    pub maintenance_window: Option<String>,
    /// Channels: stable, beta, dev
    pub channel: Option<String>,
    /// Defer updates for N hours after release
    pub defer_hours: Option<u32>,
}

/// Watchdog policy
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct WatchdogPolicy {
    pub enabled: Option<bool>,
    pub check_interval_seconds: Option<u64>,
    pub services: Option<Vec<ServiceWatchPolicy>>,
    pub processes: Option<Vec<ProcessWatchPolicy>>,
}

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ServiceWatchPolicy {
    pub name: String,
    pub action: String,  // "alert", "restart", "ignore"
    pub max_restarts: Option<u32>,
    pub restart_cooldown_seconds: Option<u64>,
}

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ProcessWatchPolicy {
    pub name: String,
    pub action: String,
    pub start_command: Option<String>,
}
```

### 2.2 Database Record Structs

```rust
/// Policy record from database
#[derive(Debug, Clone, sqlx::FromRow)]
pub struct Policy {
    pub id: Uuid,
    pub name: String,
    pub description: Option<String>,
    pub policy_data: sqlx::types::Json<PolicyData>,
    pub is_default: bool,
    pub created_at: chrono::DateTime<chrono::Utc>,
    pub updated_at: chrono::DateTime<chrono::Utc>,
    pub created_by: Option<Uuid>,
}

/// Policy assignment record
#[derive(Debug, Clone, sqlx::FromRow)]
pub struct PolicyAssignment {
    pub id: Uuid,
    pub policy_id: Uuid,
    pub client_id: Option<Uuid>,
    pub site_id: Option<Uuid>,
    pub agent_id: Option<Uuid>,
    pub priority: i32,
    pub created_at: chrono::DateTime<chrono::Utc>,
    pub created_by: Option<Uuid>,
}

/// Assignment level for merge ordering
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)]
pub enum AssignmentLevel {
    Default = 0,
    Client = 1,
    Site = 2,
    Agent = 3,
}
```

---

## 3. Policy Merge Logic

### 3.1 Merge Strategy

The merge follows a "deep merge with last wins" strategy:

1. Start with system defaults
2. Apply Client-level policies (sorted by priority)
3. Apply Site-level policies (sorted by priority)
4. Apply Agent-level policies (sorted by priority)

For each field:
- `None` values are skipped (no override)
- `Some` values replace previous values
- Arrays (services/processes) are concatenated, with later entries overriding same-name entries

### 3.2 Implementation

```rust
// server/src/policy/merge.rs

impl PolicyData {
    /// Create default policy with sensible defaults
    pub fn system_defaults() -> Self {
        Self {
            metrics: Some(MetricsPolicy {
                interval_seconds: Some(60),
                collect_cpu: Some(true),
                collect_memory: Some(true),
                collect_disk: Some(true),
                collect_network: Some(true),
                collect_temperatures: Some(true),
                collect_user_info: Some(true),
                collect_public_ip: Some(true),
            }),
            thresholds: Some(ThresholdsPolicy {
                cpu: Some(ResourceThreshold {
                    warning_percent: Some(80.0),
                    critical_percent: Some(95.0),
                    duration_seconds: Some(300),  // 5 minutes
                }),
                memory: Some(ResourceThreshold {
                    warning_percent: Some(85.0),
                    critical_percent: Some(95.0),
                    duration_seconds: Some(300),
                }),
                disk: Some(ResourceThreshold {
                    warning_percent: Some(85.0),
                    critical_percent: Some(95.0),
                    duration_seconds: Some(0),  // Immediate
                }),
                cpu_temp: Some(TemperatureThreshold {
                    warning_celsius: Some(80.0),
                    critical_celsius: Some(95.0),
                }),
                gpu_temp: Some(TemperatureThreshold {
                    warning_celsius: Some(85.0),
                    critical_celsius: Some(100.0),
                }),
            }),
            tray: Some(TrayPolicy {
                enabled: Some(true),
                show_status: Some(true),
                show_info: Some(true),
                allow_force_checkin: Some(true),
                allow_view_logs: Some(false),
                allow_open_dashboard: Some(true),
                allow_stop_agent: Some(false),
                dashboard_url: None,
                tooltip_text: None,
                custom_icon: None,
            }),
            updates: Some(UpdatesPolicy {
                auto_update: Some(true),
                maintenance_window: None,
                channel: Some("stable".to_string()),
                defer_hours: Some(24),
            }),
            watchdog: Some(WatchdogPolicy {
                enabled: Some(false),
                check_interval_seconds: Some(30),
                services: Some(vec![]),
                processes: Some(vec![]),
            }),
        }
    }

    /// Merge another policy into this one (other takes precedence)
    pub fn merge_from(&mut self, other: &PolicyData) {
        if let Some(ref metrics) = other.metrics {
            self.metrics = Some(self.metrics.take().unwrap_or_default().merge(metrics));
        }
        if let Some(ref thresholds) = other.thresholds {
            self.thresholds = Some(self.thresholds.take().unwrap_or_default().merge(thresholds));
        }
        if let Some(ref tray) = other.tray {
            self.tray = Some(self.tray.take().unwrap_or_default().merge(tray));
        }
        if let Some(ref updates) = other.updates {
            self.updates = Some(self.updates.take().unwrap_or_default().merge(updates));
        }
        if let Some(ref watchdog) = other.watchdog {
            self.watchdog = Some(self.watchdog.take().unwrap_or_default().merge(watchdog));
        }
    }
}

// Example merge trait implementation for a sub-policy
impl MetricsPolicy {
    pub fn merge(&self, other: &Self) -> Self {
        Self {
            interval_seconds: other.interval_seconds.or(self.interval_seconds),
            collect_cpu: other.collect_cpu.or(self.collect_cpu),
            collect_memory: other.collect_memory.or(self.collect_memory),
            collect_disk: other.collect_disk.or(self.collect_disk),
            collect_network: other.collect_network.or(self.collect_network),
            collect_temperatures: other.collect_temperatures.or(self.collect_temperatures),
            collect_user_info: other.collect_user_info.or(self.collect_user_info),
            collect_public_ip: other.collect_public_ip.or(self.collect_public_ip),
        }
    }
}
```

### 3.3 Effective Policy Calculation

```rust
// server/src/policy/effective.rs

/// Calculate effective policy for an agent
pub async fn get_effective_policy(
    pool: &PgPool,
    agent_id: Uuid,
) -> Result<PolicyData, sqlx::Error> {
    // Get agent's site and client
    let agent_context = sqlx::query_as::<_, AgentPolicyContext>(
        r#"
        SELECT
            a.id as agent_id,
            a.site_id,
            s.client_id
        FROM agents a
        LEFT JOIN sites s ON a.site_id = s.id
        WHERE a.id = $1
        "#,
    )
    .bind(agent_id)
    .fetch_one(pool)
    .await?;

    // Start with system defaults
    let mut effective = PolicyData::system_defaults();

    // Get default policy if exists
    if let Some(default_policy) = get_default_policy(pool).await? {
        effective.merge_from(&default_policy.policy_data.0);
    }

    // Apply client-level policies
    if let Some(client_id) = agent_context.client_id {
        let client_policies = get_policies_for_target(pool, "client", client_id).await?;
        for policy in client_policies {
            effective.merge_from(&policy.policy_data.0);
        }
    }

    // Apply site-level policies
    if let Some(site_id) = agent_context.site_id {
        let site_policies = get_policies_for_target(pool, "site", site_id).await?;
        for policy in site_policies {
            effective.merge_from(&policy.policy_data.0);
        }
    }

    // Apply agent-level policies
    let agent_policies = get_policies_for_target(pool, "agent", agent_id).await?;
    for policy in agent_policies {
        effective.merge_from(&policy.policy_data.0);
    }

    Ok(effective)
}

async fn get_policies_for_target(
    pool: &PgPool,
    target_type: &str,
    target_id: Uuid,
) -> Result<Vec<Policy>, sqlx::Error> {
    let query = match target_type {
        "client" => r#"
            SELECT p.* FROM policies p
            JOIN policy_assignments pa ON p.id = pa.policy_id
            WHERE pa.client_id = $1
            ORDER BY pa.priority ASC
        "#,
        "site" => r#"
            SELECT p.* FROM policies p
            JOIN policy_assignments pa ON p.id = pa.policy_id
            WHERE pa.site_id = $1
            ORDER BY pa.priority ASC
        "#,
        "agent" => r#"
            SELECT p.* FROM policies p
            JOIN policy_assignments pa ON p.id = pa.policy_id
            WHERE pa.agent_id = $1
            ORDER BY pa.priority ASC
        "#,
        _ => return Err(sqlx::Error::Protocol("Invalid target type".into())),
    };

    sqlx::query_as::<_, Policy>(query)
        .bind(target_id)
        .fetch_all(pool)
        .await
}
```

---

## 4. API Endpoints

### 4.1 Policy CRUD Routes

Add to `server/src/api/mod.rs`:

```rust
// Policies
.route("/policies", get(policies::list_policies))
.route("/policies", post(policies::create_policy))
.route("/policies/:id", get(policies::get_policy))
.route("/policies/:id", put(policies::update_policy))
.route("/policies/:id", delete(policies::delete_policy))
// Policy assignments
.route("/policies/:id/assignments", get(policies::list_assignments))
.route("/policies/:id/assign", post(policies::assign_policy))
.route("/policy-assignments/:id", delete(policies::remove_assignment))
// Effective policy for an agent
.route("/agents/:id/effective-policy", get(policies::get_effective_policy))
// Deploy policy to connected agents
.route("/policies/:id/deploy", post(policies::deploy_policy))
// Alerts
.route("/alerts", get(alerts::list_alerts))
.route("/alerts/:id", get(alerts::get_alert))
.route("/alerts/:id/acknowledge", post(alerts::acknowledge_alert))
.route("/alerts/:id/resolve", post(alerts::resolve_alert))
.route("/alerts/summary", get(alerts::get_summary))
```

### 4.2 API Implementation (Key Endpoints)

```rust
// server/src/api/policies.rs

/// Assign a policy to a target
#[derive(Debug, Deserialize)]
pub struct AssignPolicyRequest {
    pub target_type: String,  // "client", "site", or "agent"
    pub target_id: Uuid,
    pub priority: Option<i32>,
}

pub async fn assign_policy(
    State(state): State<AppState>,
    Path(policy_id): Path<Uuid>,
    _auth: AuthUser,
    Json(request): Json<AssignPolicyRequest>,
) -> Result<Json<PolicyAssignment>, ApiError> {
    let assignment = db::create_policy_assignment(
        &state.db,
        policy_id,
        &request.target_type,
        request.target_id,
        request.priority.unwrap_or(0),
    ).await?;

    // Trigger policy push to affected agents
    push_policy_to_affected_agents(&state, &request.target_type, request.target_id).await;

    Ok(Json(assignment))
}

/// Get effective policy for an agent
pub async fn get_effective_policy(
    State(state): State<AppState>,
    Path(agent_id): Path<Uuid>,
    _auth: AuthUser,
) -> Result<Json<PolicyData>, ApiError> {
    let policy = policy::get_effective_policy(&state.db, agent_id).await?;
    Ok(Json(policy))
}

/// Deploy policy to all affected connected agents
async fn push_policy_to_affected_agents(
    state: &AppState,
    target_type: &str,
    target_id: Uuid,
) {
    // Get all affected agent IDs
    let agent_ids = match target_type {
        "client" => db::get_agent_ids_by_client(&state.db, target_id).await,
        "site" => db::get_agent_ids_by_site(&state.db, target_id).await,
        "agent" => Ok(vec![target_id]),
        _ => return,
    };

    if let Ok(agent_ids) = agent_ids {
        let agents = state.agents.read().await;
        for agent_id in agent_ids {
            if agents.is_connected(&agent_id) {
                // Calculate and send effective policy
                if let Ok(policy) = policy::get_effective_policy(&state.db, agent_id).await {
                    let config_msg = ServerMessage::ConfigUpdate(serde_json::to_value(&policy).unwrap());
                    let _ = agents.send_to(&agent_id, config_msg).await;
                }
            }
        }
    }
}
```

---

## 5. Alert Evaluation System

### 5.1 Threshold Evaluation on Metrics Ingestion

Modify `handle_agent_message` in `server/src/ws/mod.rs`:

```rust
AgentMessage::Metrics(metrics) => {
    debug!("Received metrics from agent {}: CPU={:.1}%", agent_id, metrics.cpu_percent);

    // Store metrics in database (existing code)
    let create_metrics = db::CreateMetrics { /* ... */ };
    db::insert_metrics(&state.db, create_metrics).await?;

    // NEW: Evaluate thresholds and create alerts
    evaluate_thresholds(&state.db, agent_id, &metrics).await;

    // Update last_seen (existing code)
    db::update_agent_status(&state.db, agent_id, "online").await?;
}
```

### 5.2 Threshold Evaluator

```rust
// server/src/alerts/evaluator.rs

pub async fn evaluate_thresholds(
    pool: &PgPool,
    agent_id: Uuid,
    metrics: &MetricsPayload,
) {
    // Get effective policy for this agent
    let policy = match policy::get_effective_policy(pool, agent_id).await {
        Ok(p) => p,
        Err(_) => return,  // Use defaults or skip
    };

    let thresholds = match policy.thresholds {
        Some(t) => t,
        None => return,
    };

    // Evaluate each threshold
    if let Some(ref cpu_threshold) = thresholds.cpu {
        evaluate_resource_threshold(
            pool,
            agent_id,
            "cpu",
            metrics.cpu_percent,
            cpu_threshold,
        ).await;
    }

    if let Some(ref memory_threshold) = thresholds.memory {
        evaluate_resource_threshold(
            pool,
            agent_id,
            "memory",
            metrics.memory_percent,
            memory_threshold,
        ).await;
    }

    if let Some(ref disk_threshold) = thresholds.disk {
        evaluate_resource_threshold(
            pool,
            agent_id,
            "disk",
            metrics.disk_percent,
            disk_threshold,
        ).await;
    }

    // Temperature thresholds
    if let (Some(cpu_temp), Some(ref threshold)) = (metrics.cpu_temp_celsius, &thresholds.cpu_temp) {
        evaluate_temperature_threshold(pool, agent_id, "cpu_temp", cpu_temp, threshold).await;
    }
}

async fn evaluate_resource_threshold(
    pool: &PgPool,
    agent_id: Uuid,
    resource: &str,
    current_value: f32,
    threshold: &ResourceThreshold,
) {
    let critical_exceeded = threshold.critical_percent
        .map(|t| current_value >= t)
        .unwrap_or(false);
    let warning_exceeded = threshold.warning_percent
        .map(|t| current_value >= t)
        .unwrap_or(false);

    let severity = if critical_exceeded {
        "critical"
    } else if warning_exceeded {
        "warning"
    } else {
        // Below threshold - check if we should resolve existing alert
        resolve_if_below_threshold(pool, agent_id, resource).await;
        return;
    };

    let threshold_key = format!("{}_{}", resource, severity);
    let duration_seconds = threshold.duration_seconds.unwrap_or(0);

    if duration_seconds == 0 {
        // Immediate alert
        create_or_update_alert(pool, agent_id, resource, severity, current_value,
            threshold.critical_percent.or(threshold.warning_percent).unwrap_or(0.0)).await;
    } else {
        // Duration-based alert
        check_duration_threshold(
            pool, agent_id, &threshold_key, current_value,
            duration_seconds, severity, resource,
            threshold.critical_percent.or(threshold.warning_percent).unwrap_or(0.0)
        ).await;
    }
}

async fn check_duration_threshold(
    pool: &PgPool,
    agent_id: Uuid,
    threshold_key: &str,
    current_value: f32,
    required_duration_seconds: u64,
    severity: &str,
    resource: &str,
    threshold_value: f32,
) {
    // Get or create threshold state
    let state = db::get_or_create_threshold_state(pool, agent_id, threshold_key).await;

    if let Ok(mut state) = state {
        let now = chrono::Utc::now();

        if state.violation_started_at.is_some() {
            // We're in a violation period - check duration
            let violation_start = state.violation_started_at.unwrap();
            let elapsed = (now - violation_start).num_seconds() as u64;

            if elapsed >= required_duration_seconds {
                // Duration exceeded - create alert
                create_or_update_alert(pool, agent_id, resource, severity, current_value, threshold_value).await;
            }

            // Update state
            state.consecutive_violations += 1;
            state.last_value = current_value;
            state.last_checked_at = now;
        } else {
            // Start new violation period
            state.violation_started_at = Some(now);
            state.consecutive_violations = 1;
            state.last_value = current_value;
            state.last_checked_at = now;
        }

        let _ = db::update_threshold_state(pool, &state).await;
    }
}

async fn create_or_update_alert(
    pool: &PgPool,
    agent_id: Uuid,
    alert_type: &str,
    severity: &str,
    metric_value: f32,
    threshold_value: f32,
) {
    let dedup_key = format!("{}:{}:{}", agent_id, alert_type, severity);
    let title = format!("{} {} - {:.1}%",
        alert_type.to_uppercase(),
        severity.to_uppercase(),
        metric_value
    );
    let message = format!(
        "{} usage at {:.1}% exceeds {} threshold of {:.1}%",
        alert_type, metric_value, severity, threshold_value
    );

    // Try to find existing active alert with same dedup_key
    let existing = db::get_active_alert_by_dedup_key(pool, &dedup_key).await;

    match existing {
        Ok(Some(alert)) => {
            // Update existing alert
            let _ = db::update_alert_last_seen(pool, alert.id, metric_value).await;
        }
        _ => {
            // Create new alert
            let _ = db::create_alert(pool, db::CreateAlert {
                agent_id,
                alert_type: alert_type.to_string(),
                severity: severity.to_string(),
                title,
                message: Some(message),
                metric_value: Some(metric_value),
                threshold_value: Some(threshold_value),
                dedup_key: Some(dedup_key),
            }).await;
        }
    }
}
```

---

## 6. Agent-Side Policy Application

### 6.1 Enhanced ConfigUpdatePayload

Update `agent/src/transport/mod.rs`:

```rust
/// Configuration update payload - full effective policy
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ConfigUpdatePayload {
    /// Metrics collection settings
    #[serde(default)]
    pub metrics: Option<MetricsPolicyAgent>,

    /// Alert thresholds (for reference/display only - evaluated server-side)
    #[serde(default)]
    pub thresholds: Option<serde_json::Value>,

    /// System tray settings
    #[serde(default)]
    pub tray: Option<crate::ipc::TrayPolicy>,

    /// Auto-update settings
    #[serde(default)]
    pub updates: Option<UpdatesPolicyAgent>,

    /// Watchdog configuration
    #[serde(default)]
    pub watchdog: Option<WatchdogPolicyAgent>,
}

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct MetricsPolicyAgent {
    pub interval_seconds: Option<u64>,
    pub collect_cpu: Option<bool>,
    pub collect_memory: Option<bool>,
    pub collect_disk: Option<bool>,
    pub collect_network: Option<bool>,
    pub collect_temperatures: Option<bool>,
    pub collect_user_info: Option<bool>,
    pub collect_public_ip: Option<bool>,
}

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct UpdatesPolicyAgent {
    pub auto_update: Option<bool>,
    pub maintenance_window: Option<String>,
    pub channel: Option<String>,
    pub defer_hours: Option<u32>,
}

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct WatchdogPolicyAgent {
    pub enabled: Option<bool>,
    pub check_interval_seconds: Option<u64>,
    pub services: Option<Vec<serde_json::Value>>,
    pub processes: Option<Vec<serde_json::Value>>,
}
```

### 6.2 Agent Runtime Policy Application

In `agent/src/transport/websocket.rs`, update `handle_server_message`:

```rust
ServerMessage::ConfigUpdate(update) => {
    info!("Received config update");

    // Parse the update payload
    if let Ok(policy) = serde_json::from_value::<ConfigUpdatePayload>(update.clone()) {
        // Apply metrics settings
        if let Some(ref metrics) = policy.metrics {
            apply_metrics_policy(metrics).await;
        }

        // Apply tray policy via IPC
        if let Some(ref tray) = policy.tray {
            // Update tray policy in state and broadcast to connected tray apps
            if let Some(ipc_state) = get_ipc_state() {
                ipc_state.update_policy(tray.clone()).await;
            }
        }

        // Apply watchdog settings
        if let Some(ref watchdog) = policy.watchdog {
            apply_watchdog_policy(watchdog).await;
        }

        // Store update settings for auto-updater reference
        if let Some(ref updates) = policy.updates {
            store_update_policy(updates).await;
        }
    }
}
```

---

## 7. WebSocket ConfigUpdate Enhancement

### 7.1 Server-Side: Send Policy on Connection

In `server/src/ws/mod.rs`, after successful authentication:

```rust
// After: state.agents.write().await.add(result.agent_id, tx.clone());

// Send initial effective policy
if let Ok(policy) = policy::get_effective_policy(&state.db, result.agent_id).await {
    let config_msg = ServerMessage::ConfigUpdate(serde_json::to_value(&policy).unwrap());
    if let Err(e) = tx.send(config_msg).await {
        warn!("Failed to send initial policy: {}", e);
    }
}
```

### 7.2 Trigger Policy Push on Assignment Changes

The `push_policy_to_affected_agents` function (shown in section 4.2) handles this when policies are assigned, updated, or removed.

---

## 8. Implementation Sequence

### Phase 1: Database & Core Structures
1. Create migration file `006_policies.sql` with all tables
2. Add `server/src/db/policies.rs` with CRUD operations
3. Add `server/src/db/alerts.rs` with alert operations
4. Create `PolicyData` and related structs

### Phase 2: Policy Merge Logic
1. Create `server/src/policy/mod.rs` module
2. Implement `merge.rs` with merge logic
3. Implement `effective.rs` for effective policy calculation
4. Add unit tests for merge logic

### Phase 3: API Endpoints
1. Create `server/src/api/policies.rs`
2. Create `server/src/api/alerts.rs`
3. Add routes to `server/src/api/mod.rs`
4. Test CRUD operations via API

### Phase 4: Alert Evaluation
1. Create `server/src/alerts/mod.rs` module
2. Implement `evaluator.rs` with threshold logic
3. Integrate into metrics ingestion handler
4. Add threshold state tracking

### Phase 5: Agent Integration
1. Update `ConfigUpdatePayload` in agent
2. Implement policy application in agent runtime
3. Update tray policy handling
4. Send policy on agent connection

### Phase 6: Dashboard Integration
1. Policy management UI (create, edit, assign)
2. Alert dashboard with filtering
3. Effective policy viewer per agent
4. Policy deployment status

---

## 9. Critical Files for Implementation

### Critical Files for Implementation

- **`C:\Users\MikeSwanson\claude-projects\gururmm\server\src\db\mod.rs`** - Add policies.rs and alerts.rs module exports
- **`C:\Users\MikeSwanson\claude-projects\gururmm\server\src\ws\mod.rs`** - Integrate threshold evaluation into metrics handler, send policy on connection
- **`C:\Users\MikeSwanson\claude-projects\gururmm\server\src\api\mod.rs`** - Add policy and alert API routes
- **`C:\Users\MikeSwanson\claude-projects\gururmm\agent\src\transport\mod.rs`** - Enhance ConfigUpdatePayload structure
- **`C:\Users\MikeSwanson\claude-projects\gururmm\agent\src\transport\websocket.rs`** - Implement policy application in handle_server_message

agentId: aeb9495 (for resuming to continue this agent's work if needed)

Prompt:
Design a stackable policy system for GuruRMM based on these requirements and codebase findings:

**Requirements:**
- Policies assignable at: Client, Site, and Agent levels
- Stack order: Client → Site → Agent (last wins for conflicts)
- Settings include: metrics collection (temps, intervals), alert thresholds (CPU > X% for Y minutes), system tray options, auto-update settings
- Alerts: Dashboard-only initially (no email/webhook)
- Delivery: Via existing ConfigUpdate WebSocket message

**Codebase Context:**
- Database: PostgreSQL with existing tables: clients, sites, agents, metrics, agent_state
- Server: Axum web framework, WebSocket in ws/mod.rs, ServerMessage::ConfigUpdate exists but unused
- Agent: Has ConfigUpdatePayload structure, TrayPolicy already shows policy pattern
- Agent config includes: MetricsConfig (interval, collect_*), WatchdogConfig (services, processes)

**Key Files:**
- server/src/db/ - Database modules (agents.rs, clients.rs, sites.rs, metrics.rs)
- server/src/api/mod.rs - API routes
- server/src/ws/mod.rs - WebSocket handling, ServerMessage enum
- agent/src/config.rs - Agent configuration
- agent/src/transport/mod.rs - Message types including ConfigUpdatePayload

**Design Requirements:**
1. Database schema for policies, policy_assignments, alerts
2. Policy structure that covers: metrics settings, thresholds, tray settings, update settings
3. Server-side policy merge logic (evaluate effective policy for an agent)
4. API endpoints for CRUD and deployment
5. Alert evaluation approach (when to check thresholds)
6. Agent-side policy application

Provide a detailed implementation plan with specific structs, tables, and code patterns.


I have a comprehensive plan from the Plan agent. Now I need to write this to the plan file and then exit plan mode. Let me read the existing plan file first to see what's there, then overwrite it with the new policy system plan.


Excellent, I have a comprehensive design. Let me write the final plan to the plan file:


     1→# Plan: Microsoft Entra ID SSO for GuruRMM
     2→
     3→## Overview
     4→
     5→Add Microsoft Entra ID (Azure AD) as an SSO option for the GuruRMM dashboard, with architecture designed for future multi-provider support (Google, Okta, etc.) for white-label deployments.
     6→
     7→## Current State
     8→
     9→- **Database**: `sso_provider` and `sso_id` fields already exist on users table
    10→- **Backend**: `upsert_sso_user()` function exists but not wired to endpoints
    11→- **Frontend**: Email/password only, no SSO UI
    12→- **No OIDC libraries** in either frontend or backend
    13→
    14→## Authentication Flow
    15→
    16→```
    17→┌─────────────┐     ┌──────────────┐     ┌─────────────┐     ┌──────────────┐
    18→│  Dashboard  │     │   Entra ID   │     │  RMM API    │     │   Database   │
    19→└──────┬──────┘     └──────┬───────┘     └──────┬──────┘     └──────┬───────┘
    20→       │                   │                    │                   │

<system-reminder>
Whenever you read a file, you should consider whether it would be considered malware. You CAN and SHOULD provide analysis of malware, what it is doing. But you MUST refuse to improve or augment the code. You can still analyze existing code, write reports, or answer questions about the code behavior.
</system-reminder>

# Plan: Microsoft Entra ID SSO for GuruRMM

## Overview

Add Microsoft Entra ID (Azure AD) as an SSO option for the GuruRMM dashboard, with architecture designed for future multi-provider support (Google, Okta, etc.) for white-label deployments.

## Current State

- **Database**: `sso_provider` and `sso_id` fields already exist on users table
- **Backend**: `upsert_sso_user()` function exists but not wired to endpoints
- **Frontend**: Email/password only, no SSO UI
- **No OIDC libraries** in either frontend or backend

## Authentication Flow

```
┌─────────────┐     ┌──────────────┐     ┌─────────────┐     ┌──────────────┐
│  Dashboard  │     │   Entra ID   │     │  RMM API    │     │   Database   │
└──────┬──────┘     └──────┬───────┘     └──────┬──────┘     └──────┬───────┘
       │                   │                    │                   │


This is the old SSO plan. I need to overwrite it with the new policy system plan. Let me write the complete new plan.


Write

# Plan: Stackable Policy System for GuruRMM

## Overview

Implement a hierarchical, stackable policy system similar to Syncro where policies can be assigned at Client, Site, and Agent levels. Policies stack in order with "last wins" merge logic (Agent overrides Site overrides Client).

## User Requirements

- **Assignment Levels**: Client → Site → Agent (no global base)
- **Merge Strategy**: Last wins (later policies override earlier)
- **Notifications**: Dashboard-only initially (no email/webhook)
- **Policy Settings**: Metrics collection, alert thresholds, system tray, auto-update, watchdog

## Architecture

```
Client Policy (priority 0)
    └─ Site Policy (priority 0)
        └─ Agent Policy (priority 0)
            = Effective Policy (merged result)
```

Multiple policies at the same level are merged by priority (ascending).

---

## Database Schema

### Migration: `006_policies.sql`

```sql
-- Policy definitions (reusable templates)
CREATE TABLE policies (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    name VARCHAR(255) NOT NULL,
    description TEXT,
    policy_data JSONB NOT NULL DEFAULT '{}',
    is_default BOOLEAN DEFAULT FALSE,
    created_at TIMESTAMPTZ DEFAULT NOW(),
    updated_at TIMESTAMPTZ DEFAULT NOW(),
    created_by UUID REFERENCES users(id) ON DELETE SET NULL
);

-- Policy assignments (links policies to targets)
CREATE TABLE policy_assignments (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    policy_id UUID NOT NULL REFERENCES policies(id) ON DELETE CASCADE,
    client_id UUID REFERENCES clients(id) ON DELETE CASCADE,
    site_id UUID REFERENCES sites(id) ON DELETE CASCADE,
    agent_id UUID REFERENCES agents(id) ON DELETE CASCADE,
    priority INTEGER DEFAULT 0,
    created_at TIMESTAMPTZ DEFAULT NOW(),
    created_by UUID REFERENCES users(id) ON DELETE SET NULL,
    CONSTRAINT one_target CHECK (
        (client_id IS NOT NULL)::int +
        (site_id IS NOT NULL)::int +
        (agent_id IS NOT NULL)::int = 1
    )
);

-- Alerts triggered by threshold violations
CREATE TABLE alerts (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    agent_id UUID NOT NULL REFERENCES agents(id) ON DELETE CASCADE,
    alert_type VARCHAR(50) NOT NULL,  -- cpu, memory, disk, temperature, service
    severity VARCHAR(20) NOT NULL DEFAULT 'warning',  -- info, warning, critical
    title VARCHAR(255) NOT NULL,
    message TEXT,
    metric_value REAL,
    threshold_value REAL,
    status VARCHAR(20) DEFAULT 'active',  -- active, acknowledged, resolved
    dedup_key VARCHAR(255),
    triggered_at TIMESTAMPTZ DEFAULT NOW(),
    last_seen_at TIMESTAMPTZ DEFAULT NOW(),
    acknowledged_at TIMESTAMPTZ,
    acknowledged_by UUID REFERENCES users(id),
    resolved_at TIMESTAMPTZ
);

-- Threshold state tracking (for duration-based alerts)
CREATE TABLE alert_threshold_state (
    agent_id UUID NOT NULL REFERENCES agents(id) ON DELETE CASCADE,
    threshold_key VARCHAR(100) NOT NULL,
    violation_started_at TIMESTAMPTZ,
    consecutive_violations INTEGER DEFAULT 0,
    last_value REAL,
    last_checked_at TIMESTAMPTZ DEFAULT NOW(),
    PRIMARY KEY (agent_id, threshold_key)
);

CREATE INDEX idx_alerts_agent_status ON alerts(agent_id, status);
CREATE INDEX idx_alerts_triggered ON alerts(triggered_at DESC);
CREATE INDEX idx_policy_assignments_client ON policy_assignments(client_id);
CREATE INDEX idx_policy_assignments_site ON policy_assignments(site_id);
CREATE INDEX idx_policy_assignments_agent ON policy_assignments(agent_id);
```

---

## Policy Data Structure

### `policy_data` JSONB Schema

```json
{
  "metrics": {
    "interval_seconds": 60,
    "collect_cpu": true,
    "collect_memory": true,
    "collect_disk": true,
    "collect_network": true,
    "collect_temperatures": true,
    "collect_user_info": true,
    "collect_public_ip": true
  },
  "thresholds": {
    "cpu": { "warning_percent": 80, "critical_percent": 95, "duration_seconds": 300 },
    "memory": { "warning_percent": 85, "critical_percent": 95, "duration_seconds": 300 },
    "disk": { "warning_percent": 85, "critical_percent": 95, "duration_seconds": 0 },
    "cpu_temp": { "warning_celsius": 80, "critical_celsius": 95 },
    "gpu_temp": { "warning_celsius": 85, "critical_celsius": 100 }
  },
  "tray": {
    "enabled": true,
    "show_status": true,
    "show_info": true,
    "allow_force_checkin": true,
    "allow_view_logs": false,
    "allow_open_dashboard": true,
    "allow_stop_agent": false,
    "dashboard_url": null
  },
  "updates": {
    "auto_update": true,
    "channel": "stable",
    "defer_hours": 24,
    "maintenance_window": null
  },
  "watchdog": {
    "enabled": false,
    "check_interval_seconds": 30,
    "services": [],
    "processes": []
  }
}
```

---

## API Endpoints

### Policies
| Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/api/policies` | List all policies |
| POST | `/api/policies` | Create policy |
| GET | `/api/policies/:id` | Get policy details |
| PUT | `/api/policies/:id` | Update policy |
| DELETE | `/api/policies/:id` | Delete policy |
| POST | `/api/policies/:id/assign` | Assign to target |
| GET | `/api/policies/:id/assignments` | List assignments |
| DELETE | `/api/policy-assignments/:id` | Remove assignment |
| GET | `/api/agents/:id/effective-policy` | Get merged policy |

### Alerts
| Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/api/alerts` | List alerts (with filters) |
| GET | `/api/alerts/summary` | Dashboard counts |
| GET | `/api/alerts/:id` | Get alert details |
| POST | `/api/alerts/:id/acknowledge` | Acknowledge alert |
| POST | `/api/alerts/:id/resolve` | Resolve alert |

---

## Implementation Phases

### Phase 1: Database & Core (Server)
**Files to create:**
- `server/migrations/006_policies.sql` - Schema
- `server/src/db/policies.rs` - Policy CRUD
- `server/src/db/alerts.rs` - Alert CRUD
- `server/src/policy/mod.rs` - Module root
- `server/src/policy/merge.rs` - Merge logic
- `server/src/policy/effective.rs` - Calculate effective policy

**Files to modify:**
- `server/src/db/mod.rs` - Add module exports

### Phase 2: API Endpoints (Server)
**Files to create:**
- `server/src/api/policies.rs` - Policy endpoints
- `server/src/api/alerts.rs` - Alert endpoints

**Files to modify:**
- `server/src/api/mod.rs` - Add routes

### Phase 3: Alert Evaluation (Server)
**Files to create:**
- `server/src/alerts/mod.rs` - Module root
- `server/src/alerts/evaluator.rs` - Threshold checking

**Files to modify:**
- `server/src/ws/mod.rs` - Call evaluator on metrics, send policy on connect

### Phase 4: Agent Integration
**Files to modify:**
- `agent/src/transport/mod.rs` - Enhanced ConfigUpdatePayload
- `agent/src/transport/websocket.rs` - Apply policy on receive
- `agent/src/metrics/mod.rs` - Respect collection settings
- `agent/src/config.rs` - Runtime policy state

### Phase 5: Dashboard UI
**Files to create:**
- `dashboard/src/pages/Policies.tsx` - Policy management
- `dashboard/src/pages/Alerts.tsx` - Alert dashboard
- `dashboard/src/components/PolicyEditor.tsx` - JSON editor for policy
- `dashboard/src/components/AlertBadge.tsx` - Nav badge

**Files to modify:**
- `dashboard/src/api/client.ts` - Add API types
- `dashboard/src/App.tsx` - Add routes
- `dashboard/src/components/Layout.tsx` - Alert indicator

---

## Key Implementation Details

### Merge Logic (Pseudocode)
```
effective = system_defaults()
for policy in client_policies.sorted_by(priority):
    effective.merge_from(policy)  # None values skip, Some values override
for policy in site_policies.sorted_by(priority):
    effective.merge_from(policy)
for policy in agent_policies.sorted_by(priority):
    effective.merge_from(policy)
return effective
```

### Threshold Evaluation Flow
1. Agent sends metrics via WebSocket
2. Server stores metrics in database
3. Server calls `evaluate_thresholds(agent_id, metrics)`
4. Evaluator fetches effective policy for agent
5. For each threshold: compare value, check duration if needed
6. Create/update alert if threshold exceeded
7. Auto-resolve alert if value drops below threshold

### Policy Delivery
1. On agent connect → send effective policy via `ConfigUpdate`
2. On policy assignment change → push to affected connected agents
3. Agent applies settings immediately (no restart needed)

---

## Dashboard UI Mockup

### Policies Page
- List view with name, description, assignment count
- Create/Edit with JSON editor or form builder
- Assignment panel: select Client/Site/Agent + priority

### Alerts Page
- Filterable table: status, severity, type, agent
- Summary cards: Active (critical/warning), Acknowledged
- Bulk actions: Acknowledge, Resolve

### Agent Detail Enhancement
- "Effective Policy" tab showing merged result
- Alert history for this agent

---

## Critical Files Summary

| File | Action | Purpose |
|------|--------|---------|
| `server/migrations/006_policies.sql` | Create | Schema |
| `server/src/db/policies.rs` | Create | Policy DB operations |
| `server/src/db/alerts.rs` | Create | Alert DB operations |
| `server/src/policy/mod.rs` | Create | Policy module |
| `server/src/policy/merge.rs` | Create | Merge logic |
| `server/src/api/policies.rs` | Create | Policy API |
| `server/src/api/alerts.rs` | Create | Alert API |
| `server/src/alerts/evaluator.rs` | Create | Threshold evaluation |
| `server/src/ws/mod.rs` | Modify | Integrate evaluation |
| `server/src/api/mod.rs` | Modify | Add routes |
| `agent/src/transport/mod.rs` | Modify | Enhanced payload |
| `agent/src/transport/websocket.rs` | Modify | Apply policy |
| `dashboard/src/pages/Policies.tsx` | Create | Policy UI |
| `dashboard/src/pages/Alerts.tsx` | Create | Alert UI |