Initial GuruConnect implementation - Phase 1 MVP

- Agent: DXGI/GDI screen capture, mouse/keyboard input, WebSocket transport
- Server: Axum relay, session management, REST API
- Dashboard: React viewer components with TypeScript
- Protocol: Protobuf definitions for all message types

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
AZ Computer Guru
2025-12-21 17:18:05 -07:00
commit 33893ea73b
38 changed files with 7724 additions and 0 deletions

28
.gitignore vendored Normal file
View File

@@ -0,0 +1,28 @@
# Build artifacts
/target/
**/target/
# IDE
.idea/
.vscode/
*.swp
*.swo
# OS
.DS_Store
Thumbs.db
# Environment
.env
.env.local
*.env
# Logs
*.log
logs/
# Dependencies (if vendored)
vendor/
# Generated files
*.generated.*

117
CLAUDE.md Normal file
View File

@@ -0,0 +1,117 @@
# GuruConnect
Remote desktop solution similar to ScreenConnect, integrated with GuruRMM.
## Project Overview
GuruConnect provides remote screen control and backstage tools for Windows systems.
It's designed to be fast, secure, and enterprise-ready.
## Architecture
```
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Dashboard │◄───────►│ GuruConnect │◄───────►│ GuruConnect │
│ (React) │ WSS │ Server (Rust) │ WSS │ Agent (Rust) │
└─────────────────┘ └─────────────────┘ └─────────────────┘
```
## Directory Structure
- `agent/` - Windows remote desktop agent (Rust)
- `server/` - Relay server (Rust + Axum)
- `dashboard/` - Web viewer (React, to be integrated with GuruRMM)
- `proto/` - Protobuf protocol definitions
## Building
### Prerequisites
- Rust 1.75+ (install via rustup)
- Windows SDK (for agent)
- protoc (Protocol Buffers compiler)
### Build Commands
```bash
# Build all (from workspace root)
cargo build --release
# Build agent only
cargo build -p guruconnect-agent --release
# Build server only
cargo build -p guruconnect-server --release
```
### Cross-compilation (Agent for Windows)
From Linux build server:
```bash
# Install Windows target
rustup target add x86_64-pc-windows-msvc
# Build (requires cross or appropriate linker)
cross build -p guruconnect-agent --target x86_64-pc-windows-msvc --release
```
## Development
### Running the Server
```bash
# Development
cargo run -p guruconnect-server
# With environment variables
DATABASE_URL=postgres://... JWT_SECRET=... cargo run -p guruconnect-server
```
### Testing the Agent
The agent must be run on Windows:
```powershell
# Run from Windows
.\target\release\guruconnect-agent.exe
```
## Protocol
Uses Protocol Buffers for efficient message serialization.
See `proto/guruconnect.proto` for message definitions.
Key message types:
- `VideoFrame` - Screen frames (raw+zstd, VP9, H264)
- `MouseEvent` - Mouse input
- `KeyEvent` - Keyboard input
- `SessionRequest/Response` - Session management
## Encoding Strategy
| Scenario | Encoding |
|----------|----------|
| LAN (<20ms RTT) | Raw BGRA + Zstd + dirty rects |
| WAN + GPU | H264 hardware |
| WAN - GPU | VP9 software |
## Key References
- RustDesk source: `~/claude-projects/reference/rustdesk/`
- GuruRMM: `~/claude-projects/gururmm/`
- Plan: `~/.claude/plans/shimmering-wandering-crane.md`
## Phase 1 MVP Goals
1. DXGI screen capture with GDI fallback
2. Raw + Zstd encoding with dirty rectangle detection
3. Mouse and keyboard input injection
4. WebSocket relay through server
5. Basic React viewer
## Security Considerations
- All connections use TLS
- JWT authentication for dashboard users
- API key authentication for agents
- Session audit logging
- Optional session recording (Phase 4)

3206
Cargo.lock generated Normal file

File diff suppressed because it is too large Load Diff

27
Cargo.toml Normal file
View File

@@ -0,0 +1,27 @@
[workspace]
resolver = "2"
members = [
"agent",
"server",
]
[workspace.package]
version = "0.1.0"
edition = "2021"
authors = ["AZ Computer Guru"]
license = "Proprietary"
[workspace.dependencies]
# Shared dependencies across workspace
tokio = { version = "1", features = ["full"] }
tokio-tungstenite = { version = "0.24", features = ["native-tls"] }
prost = "0.13"
prost-types = "0.13"
bytes = "1"
serde = { version = "1", features = ["derive"] }
serde_json = "1"
tracing = "0.1"
anyhow = "1"
thiserror = "1"
uuid = { version = "1", features = ["v4", "serde"] }
chrono = { version = "0.4", features = ["serde"] }

78
agent/Cargo.toml Normal file
View File

@@ -0,0 +1,78 @@
[package]
name = "guruconnect-agent"
version = "0.1.0"
edition = "2021"
authors = ["AZ Computer Guru"]
description = "GuruConnect Remote Desktop Agent"
[dependencies]
# Async runtime
tokio = { version = "1", features = ["full", "sync", "time", "rt-multi-thread", "macros"] }
# WebSocket
tokio-tungstenite = { version = "0.24", features = ["native-tls"] }
futures-util = "0.3"
# Compression
zstd = "0.13"
# Protocol (protobuf)
prost = "0.13"
prost-types = "0.13"
bytes = "1"
# Serialization
serde = { version = "1", features = ["derive"] }
serde_json = "1"
# Logging
tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
# Error handling
anyhow = "1"
thiserror = "1"
# Configuration
toml = "0.8"
# Crypto
ring = "0.17"
# UUID
uuid = { version = "1", features = ["v4", "serde"] }
# Time
chrono = { version = "0.4", features = ["serde"] }
# Hostname
hostname = "0.4"
[target.'cfg(windows)'.dependencies]
# Windows APIs for screen capture and input
windows = { version = "0.58", features = [
"Win32_Foundation",
"Win32_Graphics_Gdi",
"Win32_Graphics_Dxgi",
"Win32_Graphics_Dxgi_Common",
"Win32_Graphics_Direct3D",
"Win32_Graphics_Direct3D11",
"Win32_UI_Input_KeyboardAndMouse",
"Win32_UI_WindowsAndMessaging",
"Win32_System_LibraryLoader",
"Win32_System_Threading",
"Win32_Security",
]}
# Windows service support
windows-service = "0.7"
[build-dependencies]
prost-build = "0.13"
[profile.release]
lto = true
codegen-units = 1
opt-level = "z"
strip = true
panic = "abort"

11
agent/build.rs Normal file
View File

@@ -0,0 +1,11 @@
use std::io::Result;
fn main() -> Result<()> {
// Compile protobuf definitions
prost_build::compile_protos(&["../proto/guruconnect.proto"], &["../proto/"])?;
// Rerun if proto changes
println!("cargo:rerun-if-changed=../proto/guruconnect.proto");
Ok(())
}

View File

@@ -0,0 +1,156 @@
//! Display enumeration and information
use anyhow::Result;
/// Information about a display/monitor
#[derive(Debug, Clone)]
pub struct Display {
/// Unique display ID
pub id: u32,
/// Display name (e.g., "\\\\.\\DISPLAY1")
pub name: String,
/// X position in virtual screen coordinates
pub x: i32,
/// Y position in virtual screen coordinates
pub y: i32,
/// Width in pixels
pub width: u32,
/// Height in pixels
pub height: u32,
/// Whether this is the primary display
pub is_primary: bool,
/// Platform-specific handle (HMONITOR on Windows)
#[cfg(windows)]
pub handle: isize,
}
/// Display info for protocol messages
#[derive(Debug, Clone)]
pub struct DisplayInfo {
pub displays: Vec<Display>,
pub primary_id: u32,
}
impl Display {
/// Total pixels in the display
pub fn pixel_count(&self) -> u32 {
self.width * self.height
}
/// Bytes needed for BGRA frame buffer
pub fn buffer_size(&self) -> usize {
(self.width * self.height * 4) as usize
}
}
/// Enumerate all connected displays
#[cfg(windows)]
pub fn enumerate_displays() -> Result<Vec<Display>> {
use windows::Win32::Graphics::Gdi::{
EnumDisplayMonitors, GetMonitorInfoW, HMONITOR, MONITORINFOEXW,
};
use windows::Win32::Foundation::{BOOL, LPARAM, RECT};
use std::mem;
let mut displays = Vec::new();
let mut display_id = 0u32;
// Callback for EnumDisplayMonitors
unsafe extern "system" fn enum_callback(
hmonitor: HMONITOR,
_hdc: windows::Win32::Graphics::Gdi::HDC,
_rect: *mut RECT,
lparam: LPARAM,
) -> BOOL {
let displays = &mut *(lparam.0 as *mut Vec<(HMONITOR, u32)>);
let id = displays.len() as u32;
displays.push((hmonitor, id));
BOOL(1) // Continue enumeration
}
// Collect all monitor handles
let mut monitors: Vec<(windows::Win32::Graphics::Gdi::HMONITOR, u32)> = Vec::new();
unsafe {
EnumDisplayMonitors(
None,
None,
Some(enum_callback),
LPARAM(&mut monitors as *mut _ as isize),
)?;
}
// Get detailed info for each monitor
for (hmonitor, id) in monitors {
let mut info: MONITORINFOEXW = unsafe { mem::zeroed() };
info.monitorInfo.cbSize = mem::size_of::<MONITORINFOEXW>() as u32;
unsafe {
if GetMonitorInfoW(hmonitor, &mut info.monitorInfo as *mut _ as *mut _).as_bool() {
let rect = info.monitorInfo.rcMonitor;
let name = String::from_utf16_lossy(
&info.szDevice[..info.szDevice.iter().position(|&c| c == 0).unwrap_or(info.szDevice.len())]
);
let is_primary = (info.monitorInfo.dwFlags & 1) != 0; // MONITORINFOF_PRIMARY
displays.push(Display {
id,
name,
x: rect.left,
y: rect.top,
width: (rect.right - rect.left) as u32,
height: (rect.bottom - rect.top) as u32,
is_primary,
handle: hmonitor.0 as isize,
});
}
}
}
// Sort by position (left to right, top to bottom)
displays.sort_by(|a, b| {
if a.y != b.y {
a.y.cmp(&b.y)
} else {
a.x.cmp(&b.x)
}
});
// Reassign IDs after sorting
for (i, display) in displays.iter_mut().enumerate() {
display.id = i as u32;
}
if displays.is_empty() {
anyhow::bail!("No displays found");
}
Ok(displays)
}
#[cfg(not(windows))]
pub fn enumerate_displays() -> Result<Vec<Display>> {
anyhow::bail!("Display enumeration only supported on Windows")
}
/// Get display info for protocol
pub fn get_display_info() -> Result<DisplayInfo> {
let displays = enumerate_displays()?;
let primary_id = displays
.iter()
.find(|d| d.is_primary)
.map(|d| d.id)
.unwrap_or(0);
Ok(DisplayInfo {
displays,
primary_id,
})
}

325
agent/src/capture/dxgi.rs Normal file
View File

@@ -0,0 +1,325 @@
//! DXGI Desktop Duplication screen capture
//!
//! Uses the Windows Desktop Duplication API (available on Windows 8+) for
//! high-performance, low-latency screen capture with hardware acceleration.
//!
//! Reference: RustDesk's scrap library implementation
use super::{CapturedFrame, Capturer, DirtyRect, Display};
use anyhow::{Context, Result};
use std::ptr;
use std::time::Instant;
use windows::Win32::Graphics::Direct3D::D3D_DRIVER_TYPE_UNKNOWN;
use windows::Win32::Graphics::Direct3D11::{
D3D11CreateDevice, ID3D11Device, ID3D11DeviceContext, ID3D11Texture2D,
D3D11_CPU_ACCESS_READ, D3D11_SDK_VERSION, D3D11_TEXTURE2D_DESC,
D3D11_USAGE_STAGING, D3D11_MAPPED_SUBRESOURCE, D3D11_MAP_READ,
};
use windows::Win32::Graphics::Dxgi::{
CreateDXGIFactory1, IDXGIAdapter1, IDXGIFactory1, IDXGIOutput, IDXGIOutput1,
IDXGIOutputDuplication, IDXGIResource, DXGI_ERROR_ACCESS_LOST,
DXGI_ERROR_WAIT_TIMEOUT, DXGI_OUTDUPL_DESC, DXGI_OUTDUPL_FRAME_INFO,
DXGI_RESOURCE_PRIORITY_MAXIMUM,
};
use windows::core::Interface;
/// DXGI Desktop Duplication capturer
pub struct DxgiCapturer {
display: Display,
device: ID3D11Device,
context: ID3D11DeviceContext,
duplication: IDXGIOutputDuplication,
staging_texture: Option<ID3D11Texture2D>,
width: u32,
height: u32,
last_frame: Option<Vec<u8>>,
}
impl DxgiCapturer {
/// Create a new DXGI capturer for the specified display
pub fn new(display: Display) -> Result<Self> {
let (device, context, duplication, desc) = Self::create_duplication(&display)?;
Ok(Self {
display,
device,
context,
duplication,
staging_texture: None,
width: desc.ModeDesc.Width,
height: desc.ModeDesc.Height,
last_frame: None,
})
}
/// Create D3D device and output duplication
fn create_duplication(
display: &Display,
) -> Result<(ID3D11Device, ID3D11DeviceContext, IDXGIOutputDuplication, DXGI_OUTDUPL_DESC)> {
unsafe {
// Create DXGI factory
let factory: IDXGIFactory1 = CreateDXGIFactory1()
.context("Failed to create DXGI factory")?;
// Find the adapter and output for this display
let (adapter, output) = Self::find_adapter_output(&factory, display)?;
// Create D3D11 device
let mut device: Option<ID3D11Device> = None;
let mut context: Option<ID3D11DeviceContext> = None;
D3D11CreateDevice(
&adapter,
D3D_DRIVER_TYPE_UNKNOWN,
None,
Default::default(),
None,
D3D11_SDK_VERSION,
Some(&mut device),
None,
Some(&mut context),
)
.context("Failed to create D3D11 device")?;
let device = device.context("D3D11 device is None")?;
let context = context.context("D3D11 context is None")?;
// Get IDXGIOutput1 interface
let output1: IDXGIOutput1 = output.cast()
.context("Failed to get IDXGIOutput1 interface")?;
// Create output duplication
let duplication = output1.DuplicateOutput(&device)
.context("Failed to create output duplication")?;
// Get duplication description
let mut desc = DXGI_OUTDUPL_DESC::default();
duplication.GetDesc(&mut desc);
tracing::info!(
"Created DXGI duplication: {}x{}, display: {}",
desc.ModeDesc.Width,
desc.ModeDesc.Height,
display.name
);
Ok((device, context, duplication, desc))
}
}
/// Find the adapter and output for the specified display
fn find_adapter_output(
factory: &IDXGIFactory1,
display: &Display,
) -> Result<(IDXGIAdapter1, IDXGIOutput)> {
unsafe {
let mut adapter_idx = 0u32;
loop {
// Enumerate adapters
let adapter: IDXGIAdapter1 = match factory.EnumAdapters1(adapter_idx) {
Ok(a) => a,
Err(_) => break,
};
let mut output_idx = 0u32;
loop {
// Enumerate outputs for this adapter
let output: IDXGIOutput = match adapter.EnumOutputs(output_idx) {
Ok(o) => o,
Err(_) => break,
};
// Check if this is the display we want
let mut desc = Default::default();
output.GetDesc(&mut desc)?;
let name = String::from_utf16_lossy(
&desc.DeviceName[..desc.DeviceName.iter().position(|&c| c == 0).unwrap_or(desc.DeviceName.len())]
);
if name == display.name || desc.Monitor.0 as isize == display.handle {
return Ok((adapter, output));
}
output_idx += 1;
}
adapter_idx += 1;
}
// If we didn't find the specific display, use the first one
let adapter: IDXGIAdapter1 = factory.EnumAdapters1(0)
.context("No adapters found")?;
let output: IDXGIOutput = adapter.EnumOutputs(0)
.context("No outputs found")?;
Ok((adapter, output))
}
}
/// Create or get the staging texture for CPU access
fn get_staging_texture(&mut self, src_texture: &ID3D11Texture2D) -> Result<&ID3D11Texture2D> {
if self.staging_texture.is_none() {
unsafe {
let mut desc = D3D11_TEXTURE2D_DESC::default();
src_texture.GetDesc(&mut desc);
desc.Usage = D3D11_USAGE_STAGING;
desc.BindFlags = Default::default();
desc.CPUAccessFlags = D3D11_CPU_ACCESS_READ;
desc.MiscFlags = Default::default();
let staging = self.device.CreateTexture2D(&desc, None)
.context("Failed to create staging texture")?;
// Set high priority
let resource: IDXGIResource = staging.cast()?;
resource.SetEvictionPriority(DXGI_RESOURCE_PRIORITY_MAXIMUM.0)?;
self.staging_texture = Some(staging);
}
}
Ok(self.staging_texture.as_ref().unwrap())
}
/// Acquire the next frame from the desktop
fn acquire_frame(&mut self, timeout_ms: u32) -> Result<Option<(ID3D11Texture2D, DXGI_OUTDUPL_FRAME_INFO)>> {
unsafe {
let mut frame_info = DXGI_OUTDUPL_FRAME_INFO::default();
let mut desktop_resource: Option<IDXGIResource> = None;
let result = self.duplication.AcquireNextFrame(
timeout_ms,
&mut frame_info,
&mut desktop_resource,
);
match result {
Ok(_) => {
let resource = desktop_resource.context("Desktop resource is None")?;
// Check if there's actually a new frame
if frame_info.LastPresentTime == 0 {
self.duplication.ReleaseFrame().ok();
return Ok(None);
}
let texture: ID3D11Texture2D = resource.cast()
.context("Failed to cast to ID3D11Texture2D")?;
Ok(Some((texture, frame_info)))
}
Err(e) if e.code() == DXGI_ERROR_WAIT_TIMEOUT => {
// No new frame available
Ok(None)
}
Err(e) if e.code() == DXGI_ERROR_ACCESS_LOST => {
// Desktop duplication was invalidated, need to recreate
tracing::warn!("Desktop duplication access lost, will need to recreate");
Err(anyhow::anyhow!("Access lost"))
}
Err(e) => {
Err(e).context("Failed to acquire frame")
}
}
}
}
/// Copy frame data to CPU-accessible memory
fn copy_frame_data(&mut self, texture: &ID3D11Texture2D) -> Result<Vec<u8>> {
unsafe {
// Get or create staging texture
let staging = self.get_staging_texture(texture)?.clone();
// Copy from GPU texture to staging texture
self.context.CopyResource(&staging, texture);
// Map the staging texture for CPU read
let mut mapped = D3D11_MAPPED_SUBRESOURCE::default();
self.context
.Map(&staging, 0, D3D11_MAP_READ, 0, Some(&mut mapped))
.context("Failed to map staging texture")?;
// Copy pixel data
let src_pitch = mapped.RowPitch as usize;
let dst_pitch = (self.width * 4) as usize;
let height = self.height as usize;
let mut data = vec![0u8; dst_pitch * height];
let src_ptr = mapped.pData as *const u8;
for y in 0..height {
let src_row = src_ptr.add(y * src_pitch);
let dst_row = data.as_mut_ptr().add(y * dst_pitch);
ptr::copy_nonoverlapping(src_row, dst_row, dst_pitch);
}
// Unmap
self.context.Unmap(&staging, 0);
Ok(data)
}
}
/// Extract dirty rectangles from frame info
fn extract_dirty_rects(&self, _frame_info: &DXGI_OUTDUPL_FRAME_INFO) -> Option<Vec<DirtyRect>> {
// TODO: Implement dirty rectangle extraction using
// IDXGIOutputDuplication::GetFrameDirtyRects and GetFrameMoveRects
// For now, return None to indicate full frame update
None
}
}
impl Capturer for DxgiCapturer {
fn capture(&mut self) -> Result<Option<CapturedFrame>> {
// Try to acquire a frame with 100ms timeout
let frame_result = self.acquire_frame(100)?;
let (texture, frame_info) = match frame_result {
Some((t, f)) => (t, f),
None => return Ok(None), // No new frame
};
// Copy frame data to CPU memory
let data = self.copy_frame_data(&texture)?;
// Release the frame
unsafe {
self.duplication.ReleaseFrame().ok();
}
// Extract dirty rectangles if available
let dirty_rects = self.extract_dirty_rects(&frame_info);
Ok(Some(CapturedFrame {
width: self.width,
height: self.height,
data,
timestamp: Instant::now(),
display_id: self.display.id,
dirty_rects,
}))
}
fn display(&self) -> &Display {
&self.display
}
fn is_valid(&self) -> bool {
// Could check if duplication is still valid
true
}
}
impl Drop for DxgiCapturer {
fn drop(&mut self) {
// Release any held frame
unsafe {
self.duplication.ReleaseFrame().ok();
}
}
}

150
agent/src/capture/gdi.rs Normal file
View File

@@ -0,0 +1,150 @@
//! GDI screen capture fallback
//!
//! Uses Windows GDI (Graphics Device Interface) for screen capture.
//! Slower than DXGI but works on older systems and edge cases.
use super::{CapturedFrame, Capturer, Display};
use anyhow::{Context, Result};
use std::time::Instant;
use windows::Win32::Graphics::Gdi::{
BitBlt, CreateCompatibleBitmap, CreateCompatibleDC, DeleteDC, DeleteObject,
GetDIBits, SelectObject, BITMAPINFO, BITMAPINFOHEADER, BI_RGB, DIB_RGB_COLORS,
SRCCOPY, GetDC, ReleaseDC,
};
use windows::Win32::Foundation::HWND;
/// GDI-based screen capturer
pub struct GdiCapturer {
display: Display,
width: u32,
height: u32,
}
impl GdiCapturer {
/// Create a new GDI capturer for the specified display
pub fn new(display: Display) -> Result<Self> {
Ok(Self {
width: display.width,
height: display.height,
display,
})
}
/// Capture the screen using GDI
fn capture_gdi(&self) -> Result<Vec<u8>> {
unsafe {
// Get device context for the entire screen
let screen_dc = GetDC(HWND::default());
if screen_dc.is_invalid() {
anyhow::bail!("Failed to get screen DC");
}
// Create compatible DC and bitmap
let mem_dc = CreateCompatibleDC(screen_dc);
if mem_dc.is_invalid() {
ReleaseDC(HWND::default(), screen_dc);
anyhow::bail!("Failed to create compatible DC");
}
let bitmap = CreateCompatibleBitmap(screen_dc, self.width as i32, self.height as i32);
if bitmap.is_invalid() {
DeleteDC(mem_dc);
ReleaseDC(HWND::default(), screen_dc);
anyhow::bail!("Failed to create compatible bitmap");
}
// Select bitmap into memory DC
let old_bitmap = SelectObject(mem_dc, bitmap);
// Copy screen to memory DC
let result = BitBlt(
mem_dc,
0,
0,
self.width as i32,
self.height as i32,
screen_dc,
self.display.x,
self.display.y,
SRCCOPY,
);
if !result.as_bool() {
SelectObject(mem_dc, old_bitmap);
DeleteObject(bitmap);
DeleteDC(mem_dc);
ReleaseDC(HWND::default(), screen_dc);
anyhow::bail!("BitBlt failed");
}
// Prepare bitmap info for GetDIBits
let mut bmi = BITMAPINFO {
bmiHeader: BITMAPINFOHEADER {
biSize: std::mem::size_of::<BITMAPINFOHEADER>() as u32,
biWidth: self.width as i32,
biHeight: -(self.height as i32), // Negative for top-down
biPlanes: 1,
biBitCount: 32,
biCompression: BI_RGB.0,
biSizeImage: 0,
biXPelsPerMeter: 0,
biYPelsPerMeter: 0,
biClrUsed: 0,
biClrImportant: 0,
},
bmiColors: [Default::default()],
};
// Allocate buffer for pixel data
let buffer_size = (self.width * self.height * 4) as usize;
let mut data = vec![0u8; buffer_size];
// Get the bits
let lines = GetDIBits(
mem_dc,
bitmap,
0,
self.height,
Some(data.as_mut_ptr() as *mut _),
&mut bmi,
DIB_RGB_COLORS,
);
// Cleanup
SelectObject(mem_dc, old_bitmap);
DeleteObject(bitmap);
DeleteDC(mem_dc);
ReleaseDC(HWND::default(), screen_dc);
if lines == 0 {
anyhow::bail!("GetDIBits failed");
}
Ok(data)
}
}
}
impl Capturer for GdiCapturer {
fn capture(&mut self) -> Result<Option<CapturedFrame>> {
let data = self.capture_gdi()?;
Ok(Some(CapturedFrame {
width: self.width,
height: self.height,
data,
timestamp: Instant::now(),
display_id: self.display.id,
dirty_rects: None, // GDI doesn't provide dirty rects
}))
}
fn display(&self) -> &Display {
&self.display
}
fn is_valid(&self) -> bool {
true
}
}

102
agent/src/capture/mod.rs Normal file
View File

@@ -0,0 +1,102 @@
//! Screen capture module
//!
//! Provides DXGI Desktop Duplication for high-performance screen capture on Windows 8+,
//! with GDI fallback for legacy systems or edge cases.
#[cfg(windows)]
mod dxgi;
#[cfg(windows)]
mod gdi;
mod display;
pub use display::{Display, DisplayInfo};
use anyhow::Result;
use std::time::Instant;
/// Captured frame data
#[derive(Debug)]
pub struct CapturedFrame {
/// Frame width in pixels
pub width: u32,
/// Frame height in pixels
pub height: u32,
/// Raw BGRA pixel data (4 bytes per pixel)
pub data: Vec<u8>,
/// Timestamp when frame was captured
pub timestamp: Instant,
/// Display ID this frame is from
pub display_id: u32,
/// Regions that changed since last frame (if available)
pub dirty_rects: Option<Vec<DirtyRect>>,
}
/// Rectangular region that changed
#[derive(Debug, Clone, Copy)]
pub struct DirtyRect {
pub x: u32,
pub y: u32,
pub width: u32,
pub height: u32,
}
/// Screen capturer trait
pub trait Capturer: Send {
/// Capture the next frame
///
/// Returns None if no new frame is available (screen unchanged)
fn capture(&mut self) -> Result<Option<CapturedFrame>>;
/// Get the current display info
fn display(&self) -> &Display;
/// Check if capturer is still valid (display may have changed)
fn is_valid(&self) -> bool;
}
/// Create a capturer for the specified display
#[cfg(windows)]
pub fn create_capturer(display: Display, use_dxgi: bool, gdi_fallback: bool) -> Result<Box<dyn Capturer>> {
if use_dxgi {
match dxgi::DxgiCapturer::new(display.clone()) {
Ok(capturer) => {
tracing::info!("Using DXGI Desktop Duplication for capture");
return Ok(Box::new(capturer));
}
Err(e) => {
tracing::warn!("DXGI capture failed: {}, trying fallback", e);
if !gdi_fallback {
return Err(e);
}
}
}
}
// GDI fallback
tracing::info!("Using GDI for capture");
Ok(Box::new(gdi::GdiCapturer::new(display)?))
}
#[cfg(not(windows))]
pub fn create_capturer(_display: Display, _use_dxgi: bool, _gdi_fallback: bool) -> Result<Box<dyn Capturer>> {
anyhow::bail!("Screen capture only supported on Windows")
}
/// Get all available displays
pub fn enumerate_displays() -> Result<Vec<Display>> {
display::enumerate_displays()
}
/// Get the primary display
pub fn primary_display() -> Result<Display> {
let displays = enumerate_displays()?;
displays
.into_iter()
.find(|d| d.is_primary)
.ok_or_else(|| anyhow::anyhow!("No primary display found"))
}

199
agent/src/config.rs Normal file
View File

@@ -0,0 +1,199 @@
//! Agent configuration management
use anyhow::{Context, Result};
use serde::{Deserialize, Serialize};
use std::path::PathBuf;
/// Agent configuration
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Config {
/// Server WebSocket URL (e.g., wss://connect.example.com/ws)
pub server_url: String,
/// Agent API key for authentication
pub api_key: String,
/// Optional hostname override
pub hostname_override: Option<String>,
/// Capture settings
#[serde(default)]
pub capture: CaptureConfig,
/// Encoding settings
#[serde(default)]
pub encoding: EncodingConfig,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct CaptureConfig {
/// Target frames per second (1-60)
#[serde(default = "default_fps")]
pub fps: u32,
/// Use DXGI Desktop Duplication (recommended)
#[serde(default = "default_true")]
pub use_dxgi: bool,
/// Fall back to GDI if DXGI fails
#[serde(default = "default_true")]
pub gdi_fallback: bool,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct EncodingConfig {
/// Preferred codec (auto, raw, vp9, h264)
#[serde(default = "default_codec")]
pub codec: String,
/// Quality (1-100, higher = better quality, more bandwidth)
#[serde(default = "default_quality")]
pub quality: u32,
/// Use hardware encoding if available
#[serde(default = "default_true")]
pub hardware_encoding: bool,
}
fn default_fps() -> u32 {
30
}
fn default_true() -> bool {
true
}
fn default_codec() -> String {
"auto".to_string()
}
fn default_quality() -> u32 {
75
}
impl Default for CaptureConfig {
fn default() -> Self {
Self {
fps: default_fps(),
use_dxgi: true,
gdi_fallback: true,
}
}
}
impl Default for EncodingConfig {
fn default() -> Self {
Self {
codec: default_codec(),
quality: default_quality(),
hardware_encoding: true,
}
}
}
impl Config {
/// Load configuration from file or environment
pub fn load() -> Result<Self> {
// Try loading from config file
let config_path = Self::config_path();
if config_path.exists() {
let contents = std::fs::read_to_string(&config_path)
.with_context(|| format!("Failed to read config from {:?}", config_path))?;
let config: Config = toml::from_str(&contents)
.with_context(|| "Failed to parse config file")?;
return Ok(config);
}
// Fall back to environment variables
let server_url = std::env::var("GURUCONNECT_SERVER_URL")
.unwrap_or_else(|_| "wss://localhost:3002/ws".to_string());
let api_key = std::env::var("GURUCONNECT_API_KEY")
.unwrap_or_else(|_| "dev-key".to_string());
Ok(Config {
server_url,
api_key,
hostname_override: std::env::var("GURUCONNECT_HOSTNAME").ok(),
capture: CaptureConfig::default(),
encoding: EncodingConfig::default(),
})
}
/// Get the configuration file path
fn config_path() -> PathBuf {
// Check for config in current directory first
let local_config = PathBuf::from("guruconnect.toml");
if local_config.exists() {
return local_config;
}
// Check in program data directory (Windows)
#[cfg(windows)]
{
if let Ok(program_data) = std::env::var("ProgramData") {
let path = PathBuf::from(program_data)
.join("GuruConnect")
.join("agent.toml");
if path.exists() {
return path;
}
}
}
// Default to local config
local_config
}
/// Get the hostname to use
pub fn hostname(&self) -> String {
self.hostname_override
.clone()
.unwrap_or_else(|| {
hostname::get()
.map(|h| h.to_string_lossy().to_string())
.unwrap_or_else(|_| "unknown".to_string())
})
}
/// Save current configuration to file
pub fn save(&self) -> Result<()> {
let config_path = Self::config_path();
// Ensure parent directory exists
if let Some(parent) = config_path.parent() {
std::fs::create_dir_all(parent)?;
}
let contents = toml::to_string_pretty(self)?;
std::fs::write(&config_path, contents)?;
Ok(())
}
}
/// Example configuration file content
pub fn example_config() -> &'static str {
r#"# GuruConnect Agent Configuration
# Server connection
server_url = "wss://connect.example.com/ws"
api_key = "your-agent-api-key"
# Optional: override hostname
# hostname_override = "custom-hostname"
[capture]
fps = 30
use_dxgi = true
gdi_fallback = true
[encoding]
codec = "auto" # auto, raw, vp9, h264
quality = 75 # 1-100
hardware_encoding = true
"#
}

52
agent/src/encoder/mod.rs Normal file
View File

@@ -0,0 +1,52 @@
//! Frame encoding module
//!
//! Encodes captured frames for transmission. Supports:
//! - Raw BGRA + Zstd compression (lowest latency, LAN mode)
//! - VP9 software encoding (universal fallback)
//! - H264 hardware encoding (when GPU available)
mod raw;
pub use raw::RawEncoder;
use crate::capture::CapturedFrame;
use crate::proto::{VideoFrame, RawFrame, DirtyRect as ProtoDirtyRect};
use anyhow::Result;
/// Encoded frame ready for transmission
#[derive(Debug)]
pub struct EncodedFrame {
/// Protobuf video frame message
pub frame: VideoFrame,
/// Size in bytes after encoding
pub size: usize,
/// Whether this is a keyframe (full frame)
pub is_keyframe: bool,
}
/// Frame encoder trait
pub trait Encoder: Send {
/// Encode a captured frame
fn encode(&mut self, frame: &CapturedFrame) -> Result<EncodedFrame>;
/// Request a keyframe on next encode
fn request_keyframe(&mut self);
/// Get encoder name/type
fn name(&self) -> &str;
}
/// Create an encoder based on configuration
pub fn create_encoder(codec: &str, quality: u32) -> Result<Box<dyn Encoder>> {
match codec.to_lowercase().as_str() {
"raw" | "zstd" => Ok(Box::new(RawEncoder::new(quality)?)),
// "vp9" => Ok(Box::new(Vp9Encoder::new(quality)?)),
// "h264" => Ok(Box::new(H264Encoder::new(quality)?)),
"auto" | _ => {
// Default to raw for now (best for LAN)
Ok(Box::new(RawEncoder::new(quality)?))
}
}
}

232
agent/src/encoder/raw.rs Normal file
View File

@@ -0,0 +1,232 @@
//! Raw frame encoder with Zstd compression
//!
//! Best for LAN connections where bandwidth is plentiful and latency is critical.
//! Compresses BGRA pixel data using Zstd for fast compression/decompression.
use super::{EncodedFrame, Encoder};
use crate::capture::{CapturedFrame, DirtyRect};
use crate::proto::{video_frame, DirtyRect as ProtoDirtyRect, RawFrame, VideoFrame};
use anyhow::Result;
/// Raw frame encoder with Zstd compression
pub struct RawEncoder {
/// Compression level (1-22, default 3 for speed)
compression_level: i32,
/// Previous frame for delta detection
previous_frame: Option<Vec<u8>>,
/// Force keyframe on next encode
force_keyframe: bool,
/// Frame counter
sequence: u32,
}
impl RawEncoder {
/// Create a new raw encoder
///
/// Quality 1-100 maps to Zstd compression level:
/// - Low quality (1-33): Level 1-3 (fastest)
/// - Medium quality (34-66): Level 4-9
/// - High quality (67-100): Level 10-15 (best compression)
pub fn new(quality: u32) -> Result<Self> {
let compression_level = Self::quality_to_level(quality);
Ok(Self {
compression_level,
previous_frame: None,
force_keyframe: true, // Start with keyframe
sequence: 0,
})
}
/// Convert quality (1-100) to Zstd compression level
fn quality_to_level(quality: u32) -> i32 {
// Lower quality = faster compression (level 1-3)
// Higher quality = better compression (level 10-15)
// We optimize for speed, so cap at 6
match quality {
0..=33 => 1,
34..=50 => 2,
51..=66 => 3,
67..=80 => 4,
81..=90 => 5,
_ => 6,
}
}
/// Compress data using Zstd
fn compress(&self, data: &[u8]) -> Result<Vec<u8>> {
let compressed = zstd::encode_all(data, self.compression_level)?;
Ok(compressed)
}
/// Detect dirty rectangles by comparing with previous frame
fn detect_dirty_rects(
&self,
current: &[u8],
previous: &[u8],
width: u32,
height: u32,
) -> Vec<DirtyRect> {
// Simple block-based dirty detection
// Divide screen into 64x64 blocks and check which changed
const BLOCK_SIZE: u32 = 64;
let mut dirty_rects = Vec::new();
let stride = (width * 4) as usize;
let blocks_x = (width + BLOCK_SIZE - 1) / BLOCK_SIZE;
let blocks_y = (height + BLOCK_SIZE - 1) / BLOCK_SIZE;
for by in 0..blocks_y {
for bx in 0..blocks_x {
let x = bx * BLOCK_SIZE;
let y = by * BLOCK_SIZE;
let block_w = (BLOCK_SIZE).min(width - x);
let block_h = (BLOCK_SIZE).min(height - y);
// Check if this block changed
let mut changed = false;
'block_check: for row in 0..block_h {
let row_start = ((y + row) as usize * stride) + (x as usize * 4);
let row_end = row_start + (block_w as usize * 4);
if row_end <= current.len() && row_end <= previous.len() {
if current[row_start..row_end] != previous[row_start..row_end] {
changed = true;
break 'block_check;
}
} else {
changed = true;
break 'block_check;
}
}
if changed {
dirty_rects.push(DirtyRect {
x,
y,
width: block_w,
height: block_h,
});
}
}
}
// Merge adjacent dirty rects (simple optimization)
// TODO: Implement proper rectangle merging
dirty_rects
}
/// Extract pixels for dirty rectangles only
fn extract_dirty_pixels(
&self,
data: &[u8],
width: u32,
dirty_rects: &[DirtyRect],
) -> Vec<u8> {
let stride = (width * 4) as usize;
let mut pixels = Vec::new();
for rect in dirty_rects {
for row in 0..rect.height {
let row_start = ((rect.y + row) as usize * stride) + (rect.x as usize * 4);
let row_end = row_start + (rect.width as usize * 4);
if row_end <= data.len() {
pixels.extend_from_slice(&data[row_start..row_end]);
}
}
}
pixels
}
}
impl Encoder for RawEncoder {
fn encode(&mut self, frame: &CapturedFrame) -> Result<EncodedFrame> {
self.sequence = self.sequence.wrapping_add(1);
let is_keyframe = self.force_keyframe || self.previous_frame.is_none();
self.force_keyframe = false;
let (data_to_compress, dirty_rects, full_frame) = if is_keyframe {
// Keyframe: send full frame
(frame.data.clone(), Vec::new(), true)
} else if let Some(ref previous) = self.previous_frame {
// Delta frame: detect and send only changed regions
let dirty_rects =
self.detect_dirty_rects(&frame.data, previous, frame.width, frame.height);
if dirty_rects.is_empty() {
// No changes, skip frame
return Ok(EncodedFrame {
frame: VideoFrame::default(),
size: 0,
is_keyframe: false,
});
}
// If too many dirty rects, just send full frame
if dirty_rects.len() > 50 {
(frame.data.clone(), Vec::new(), true)
} else {
let dirty_pixels = self.extract_dirty_pixels(&frame.data, frame.width, &dirty_rects);
(dirty_pixels, dirty_rects, false)
}
} else {
(frame.data.clone(), Vec::new(), true)
};
// Compress the data
let compressed = self.compress(&data_to_compress)?;
let size = compressed.len();
// Build protobuf message
let proto_dirty_rects: Vec<ProtoDirtyRect> = dirty_rects
.iter()
.map(|r| ProtoDirtyRect {
x: r.x as i32,
y: r.y as i32,
width: r.width as i32,
height: r.height as i32,
})
.collect();
let raw_frame = RawFrame {
width: frame.width as i32,
height: frame.height as i32,
data: compressed,
compressed: true,
dirty_rects: proto_dirty_rects,
is_keyframe: full_frame,
};
let video_frame = VideoFrame {
timestamp: frame.timestamp.elapsed().as_millis() as i64,
display_id: frame.display_id as i32,
sequence: self.sequence as i32,
encoding: Some(video_frame::Encoding::Raw(raw_frame)),
};
// Save current frame for next comparison
self.previous_frame = Some(frame.data.clone());
Ok(EncodedFrame {
frame: video_frame,
size,
is_keyframe: full_frame,
})
}
fn request_keyframe(&mut self) {
self.force_keyframe = true;
}
fn name(&self) -> &str {
"raw+zstd"
}
}

287
agent/src/input/keyboard.rs Normal file
View File

@@ -0,0 +1,287 @@
//! Keyboard input simulation using Windows SendInput API
use anyhow::Result;
#[cfg(windows)]
use windows::Win32::UI::Input::KeyboardAndMouse::{
SendInput, INPUT, INPUT_0, INPUT_KEYBOARD, KEYBD_EVENT_FLAGS, KEYEVENTF_EXTENDEDKEY,
KEYEVENTF_KEYUP, KEYEVENTF_SCANCODE, KEYEVENTF_UNICODE, KEYBDINPUT,
MapVirtualKeyW, MAPVK_VK_TO_VSC_EX,
};
/// Keyboard input controller
pub struct KeyboardController {
// Track modifier states for proper handling
#[allow(dead_code)]
modifiers: ModifierState,
}
#[derive(Default)]
struct ModifierState {
ctrl: bool,
alt: bool,
shift: bool,
meta: bool,
}
impl KeyboardController {
/// Create a new keyboard controller
pub fn new() -> Result<Self> {
Ok(Self {
modifiers: ModifierState::default(),
})
}
/// Press a key down by virtual key code
#[cfg(windows)]
pub fn key_down(&mut self, vk_code: u16) -> Result<()> {
self.send_key(vk_code, true)
}
/// Release a key by virtual key code
#[cfg(windows)]
pub fn key_up(&mut self, vk_code: u16) -> Result<()> {
self.send_key(vk_code, false)
}
/// Send a key event
#[cfg(windows)]
fn send_key(&mut self, vk_code: u16, down: bool) -> Result<()> {
// Get scan code from virtual key
let scan_code = unsafe { MapVirtualKeyW(vk_code as u32, MAPVK_VK_TO_VSC_EX) as u16 };
let mut flags = KEYBD_EVENT_FLAGS::default();
// Add extended key flag for certain keys
if Self::is_extended_key(vk_code) || (scan_code >> 8) == 0xE0 {
flags |= KEYEVENTF_EXTENDEDKEY;
}
if !down {
flags |= KEYEVENTF_KEYUP;
}
let input = INPUT {
r#type: INPUT_KEYBOARD,
Anonymous: INPUT_0 {
ki: KEYBDINPUT {
wVk: windows::Win32::UI::Input::KeyboardAndMouse::VIRTUAL_KEY(vk_code),
wScan: scan_code,
dwFlags: flags,
time: 0,
dwExtraInfo: 0,
},
},
};
self.send_input(&[input])
}
/// Type a unicode character
#[cfg(windows)]
pub fn type_char(&mut self, ch: char) -> Result<()> {
let mut inputs = Vec::new();
// For characters that fit in a single u16
for code_unit in ch.encode_utf16(&mut [0; 2]) {
// Key down
inputs.push(INPUT {
r#type: INPUT_KEYBOARD,
Anonymous: INPUT_0 {
ki: KEYBDINPUT {
wVk: windows::Win32::UI::Input::KeyboardAndMouse::VIRTUAL_KEY(0),
wScan: code_unit,
dwFlags: KEYEVENTF_UNICODE,
time: 0,
dwExtraInfo: 0,
},
},
});
// Key up
inputs.push(INPUT {
r#type: INPUT_KEYBOARD,
Anonymous: INPUT_0 {
ki: KEYBDINPUT {
wVk: windows::Win32::UI::Input::KeyboardAndMouse::VIRTUAL_KEY(0),
wScan: code_unit,
dwFlags: KEYEVENTF_UNICODE | KEYEVENTF_KEYUP,
time: 0,
dwExtraInfo: 0,
},
},
});
}
self.send_input(&inputs)
}
/// Type a string of text
#[cfg(windows)]
pub fn type_string(&mut self, text: &str) -> Result<()> {
for ch in text.chars() {
self.type_char(ch)?;
}
Ok(())
}
/// Send Secure Attention Sequence (Ctrl+Alt+Delete)
///
/// Note: This requires special privileges on Windows.
/// The agent typically needs to run as SYSTEM or use SAS API.
#[cfg(windows)]
pub fn send_sas(&mut self) -> Result<()> {
// Try using the SAS library if available
// For now, we'll attempt to send the key combination
// This won't work in all contexts due to Windows security
// Load the sas.dll and call SendSAS if available
use windows::Win32::System::LibraryLoader::{GetProcAddress, LoadLibraryW};
use windows::core::PCWSTR;
unsafe {
let dll_name: Vec<u16> = "sas.dll\0".encode_utf16().collect();
let lib = LoadLibraryW(PCWSTR(dll_name.as_ptr()));
if let Ok(lib) = lib {
let proc_name = b"SendSAS\0";
if let Some(proc) = GetProcAddress(lib, windows::core::PCSTR(proc_name.as_ptr())) {
// SendSAS takes a BOOL parameter: FALSE for Ctrl+Alt+Del
let send_sas: extern "system" fn(i32) = std::mem::transmute(proc);
send_sas(0); // FALSE = Ctrl+Alt+Del
return Ok(());
}
}
}
// Fallback: Try sending the keys (won't work without proper privileges)
tracing::warn!("SAS library not available, Ctrl+Alt+Del may not work");
// VK codes
const VK_CONTROL: u16 = 0x11;
const VK_MENU: u16 = 0x12; // Alt
const VK_DELETE: u16 = 0x2E;
// Press keys
self.key_down(VK_CONTROL)?;
self.key_down(VK_MENU)?;
self.key_down(VK_DELETE)?;
// Release keys
self.key_up(VK_DELETE)?;
self.key_up(VK_MENU)?;
self.key_up(VK_CONTROL)?;
Ok(())
}
/// Check if a virtual key code is an extended key
#[cfg(windows)]
fn is_extended_key(vk: u16) -> bool {
matches!(
vk,
0x21..=0x28 | // Page Up, Page Down, End, Home, Arrow keys
0x2D | 0x2E | // Insert, Delete
0x5B | 0x5C | // Left/Right Windows keys
0x5D | // Applications key
0x6F | // Numpad Divide
0x90 | // Num Lock
0x91 // Scroll Lock
)
}
/// Send input events
#[cfg(windows)]
fn send_input(&self, inputs: &[INPUT]) -> Result<()> {
let sent = unsafe { SendInput(inputs, std::mem::size_of::<INPUT>() as i32) };
if sent as usize != inputs.len() {
anyhow::bail!(
"SendInput failed: sent {} of {} inputs",
sent,
inputs.len()
);
}
Ok(())
}
#[cfg(not(windows))]
pub fn key_down(&mut self, _vk_code: u16) -> Result<()> {
anyhow::bail!("Keyboard input only supported on Windows")
}
#[cfg(not(windows))]
pub fn key_up(&mut self, _vk_code: u16) -> Result<()> {
anyhow::bail!("Keyboard input only supported on Windows")
}
#[cfg(not(windows))]
pub fn type_char(&mut self, _ch: char) -> Result<()> {
anyhow::bail!("Keyboard input only supported on Windows")
}
#[cfg(not(windows))]
pub fn send_sas(&mut self) -> Result<()> {
anyhow::bail!("SAS only supported on Windows")
}
}
/// Common Windows virtual key codes
#[allow(dead_code)]
pub mod vk {
pub const BACK: u16 = 0x08;
pub const TAB: u16 = 0x09;
pub const RETURN: u16 = 0x0D;
pub const SHIFT: u16 = 0x10;
pub const CONTROL: u16 = 0x11;
pub const MENU: u16 = 0x12; // Alt
pub const PAUSE: u16 = 0x13;
pub const CAPITAL: u16 = 0x14; // Caps Lock
pub const ESCAPE: u16 = 0x1B;
pub const SPACE: u16 = 0x20;
pub const PRIOR: u16 = 0x21; // Page Up
pub const NEXT: u16 = 0x22; // Page Down
pub const END: u16 = 0x23;
pub const HOME: u16 = 0x24;
pub const LEFT: u16 = 0x25;
pub const UP: u16 = 0x26;
pub const RIGHT: u16 = 0x27;
pub const DOWN: u16 = 0x28;
pub const INSERT: u16 = 0x2D;
pub const DELETE: u16 = 0x2E;
// 0-9 keys
pub const KEY_0: u16 = 0x30;
pub const KEY_9: u16 = 0x39;
// A-Z keys
pub const KEY_A: u16 = 0x41;
pub const KEY_Z: u16 = 0x5A;
// Windows keys
pub const LWIN: u16 = 0x5B;
pub const RWIN: u16 = 0x5C;
// Function keys
pub const F1: u16 = 0x70;
pub const F2: u16 = 0x71;
pub const F3: u16 = 0x72;
pub const F4: u16 = 0x73;
pub const F5: u16 = 0x74;
pub const F6: u16 = 0x75;
pub const F7: u16 = 0x76;
pub const F8: u16 = 0x77;
pub const F9: u16 = 0x78;
pub const F10: u16 = 0x79;
pub const F11: u16 = 0x7A;
pub const F12: u16 = 0x7B;
// Modifier keys
pub const LSHIFT: u16 = 0xA0;
pub const RSHIFT: u16 = 0xA1;
pub const LCONTROL: u16 = 0xA2;
pub const RCONTROL: u16 = 0xA3;
pub const LMENU: u16 = 0xA4; // Left Alt
pub const RMENU: u16 = 0xA5; // Right Alt
}

91
agent/src/input/mod.rs Normal file
View File

@@ -0,0 +1,91 @@
//! Input injection module
//!
//! Handles mouse and keyboard input simulation using Windows SendInput API.
mod mouse;
mod keyboard;
pub use mouse::MouseController;
pub use keyboard::KeyboardController;
use anyhow::Result;
/// Combined input controller for mouse and keyboard
pub struct InputController {
mouse: MouseController,
keyboard: KeyboardController,
}
impl InputController {
/// Create a new input controller
pub fn new() -> Result<Self> {
Ok(Self {
mouse: MouseController::new()?,
keyboard: KeyboardController::new()?,
})
}
/// Get mouse controller
pub fn mouse(&mut self) -> &mut MouseController {
&mut self.mouse
}
/// Get keyboard controller
pub fn keyboard(&mut self) -> &mut KeyboardController {
&mut self.keyboard
}
/// Move mouse to absolute position
pub fn mouse_move(&mut self, x: i32, y: i32) -> Result<()> {
self.mouse.move_to(x, y)
}
/// Click mouse button
pub fn mouse_click(&mut self, button: MouseButton, down: bool) -> Result<()> {
if down {
self.mouse.button_down(button)
} else {
self.mouse.button_up(button)
}
}
/// Scroll mouse wheel
pub fn mouse_scroll(&mut self, delta_x: i32, delta_y: i32) -> Result<()> {
self.mouse.scroll(delta_x, delta_y)
}
/// Press or release a key
pub fn key_event(&mut self, vk_code: u16, down: bool) -> Result<()> {
if down {
self.keyboard.key_down(vk_code)
} else {
self.keyboard.key_up(vk_code)
}
}
/// Type a unicode character
pub fn type_unicode(&mut self, ch: char) -> Result<()> {
self.keyboard.type_char(ch)
}
/// Send Ctrl+Alt+Delete (requires special handling on Windows)
pub fn send_ctrl_alt_del(&mut self) -> Result<()> {
self.keyboard.send_sas()
}
}
/// Mouse button types
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum MouseButton {
Left,
Right,
Middle,
X1,
X2,
}
impl Default for InputController {
fn default() -> Self {
Self::new().expect("Failed to create input controller")
}
}

217
agent/src/input/mouse.rs Normal file
View File

@@ -0,0 +1,217 @@
//! Mouse input simulation using Windows SendInput API
use super::MouseButton;
use anyhow::Result;
#[cfg(windows)]
use windows::Win32::UI::Input::KeyboardAndMouse::{
SendInput, INPUT, INPUT_0, INPUT_MOUSE, MOUSEEVENTF_ABSOLUTE, MOUSEEVENTF_HWHEEL,
MOUSEEVENTF_LEFTDOWN, MOUSEEVENTF_LEFTUP, MOUSEEVENTF_MIDDLEDOWN, MOUSEEVENTF_MIDDLEUP,
MOUSEEVENTF_MOVE, MOUSEEVENTF_RIGHTDOWN, MOUSEEVENTF_RIGHTUP, MOUSEEVENTF_VIRTUALDESK,
MOUSEEVENTF_WHEEL, MOUSEEVENTF_XDOWN, MOUSEEVENTF_XUP, MOUSEINPUT, XBUTTON1, XBUTTON2,
};
#[cfg(windows)]
use windows::Win32::UI::WindowsAndMessaging::{
GetSystemMetrics, SM_CXVIRTUALSCREEN, SM_CYVIRTUALSCREEN, SM_XVIRTUALSCREEN,
SM_YVIRTUALSCREEN,
};
/// Mouse input controller
pub struct MouseController {
/// Virtual screen dimensions for coordinate translation
#[cfg(windows)]
virtual_screen: VirtualScreen,
}
#[cfg(windows)]
struct VirtualScreen {
x: i32,
y: i32,
width: i32,
height: i32,
}
impl MouseController {
/// Create a new mouse controller
pub fn new() -> Result<Self> {
#[cfg(windows)]
{
let virtual_screen = unsafe {
VirtualScreen {
x: GetSystemMetrics(SM_XVIRTUALSCREEN),
y: GetSystemMetrics(SM_YVIRTUALSCREEN),
width: GetSystemMetrics(SM_CXVIRTUALSCREEN),
height: GetSystemMetrics(SM_CYVIRTUALSCREEN),
}
};
Ok(Self { virtual_screen })
}
#[cfg(not(windows))]
{
anyhow::bail!("Mouse input only supported on Windows")
}
}
/// Move mouse to absolute screen coordinates
#[cfg(windows)]
pub fn move_to(&mut self, x: i32, y: i32) -> Result<()> {
// Convert screen coordinates to normalized absolute coordinates (0-65535)
let norm_x = ((x - self.virtual_screen.x) * 65535) / self.virtual_screen.width;
let norm_y = ((y - self.virtual_screen.y) * 65535) / self.virtual_screen.height;
let input = INPUT {
r#type: INPUT_MOUSE,
Anonymous: INPUT_0 {
mi: MOUSEINPUT {
dx: norm_x,
dy: norm_y,
mouseData: 0,
dwFlags: MOUSEEVENTF_MOVE | MOUSEEVENTF_ABSOLUTE | MOUSEEVENTF_VIRTUALDESK,
time: 0,
dwExtraInfo: 0,
},
},
};
self.send_input(&[input])
}
/// Press mouse button down
#[cfg(windows)]
pub fn button_down(&mut self, button: MouseButton) -> Result<()> {
let (flags, data) = match button {
MouseButton::Left => (MOUSEEVENTF_LEFTDOWN, 0),
MouseButton::Right => (MOUSEEVENTF_RIGHTDOWN, 0),
MouseButton::Middle => (MOUSEEVENTF_MIDDLEDOWN, 0),
MouseButton::X1 => (MOUSEEVENTF_XDOWN, XBUTTON1 as u32),
MouseButton::X2 => (MOUSEEVENTF_XDOWN, XBUTTON2 as u32),
};
let input = INPUT {
r#type: INPUT_MOUSE,
Anonymous: INPUT_0 {
mi: MOUSEINPUT {
dx: 0,
dy: 0,
mouseData: data,
dwFlags: flags,
time: 0,
dwExtraInfo: 0,
},
},
};
self.send_input(&[input])
}
/// Release mouse button
#[cfg(windows)]
pub fn button_up(&mut self, button: MouseButton) -> Result<()> {
let (flags, data) = match button {
MouseButton::Left => (MOUSEEVENTF_LEFTUP, 0),
MouseButton::Right => (MOUSEEVENTF_RIGHTUP, 0),
MouseButton::Middle => (MOUSEEVENTF_MIDDLEUP, 0),
MouseButton::X1 => (MOUSEEVENTF_XUP, XBUTTON1 as u32),
MouseButton::X2 => (MOUSEEVENTF_XUP, XBUTTON2 as u32),
};
let input = INPUT {
r#type: INPUT_MOUSE,
Anonymous: INPUT_0 {
mi: MOUSEINPUT {
dx: 0,
dy: 0,
mouseData: data,
dwFlags: flags,
time: 0,
dwExtraInfo: 0,
},
},
};
self.send_input(&[input])
}
/// Scroll mouse wheel
#[cfg(windows)]
pub fn scroll(&mut self, delta_x: i32, delta_y: i32) -> Result<()> {
let mut inputs = Vec::new();
// Vertical scroll
if delta_y != 0 {
inputs.push(INPUT {
r#type: INPUT_MOUSE,
Anonymous: INPUT_0 {
mi: MOUSEINPUT {
dx: 0,
dy: 0,
mouseData: delta_y as u32,
dwFlags: MOUSEEVENTF_WHEEL,
time: 0,
dwExtraInfo: 0,
},
},
});
}
// Horizontal scroll
if delta_x != 0 {
inputs.push(INPUT {
r#type: INPUT_MOUSE,
Anonymous: INPUT_0 {
mi: MOUSEINPUT {
dx: 0,
dy: 0,
mouseData: delta_x as u32,
dwFlags: MOUSEEVENTF_HWHEEL,
time: 0,
dwExtraInfo: 0,
},
},
});
}
if !inputs.is_empty() {
self.send_input(&inputs)?;
}
Ok(())
}
/// Send input events
#[cfg(windows)]
fn send_input(&self, inputs: &[INPUT]) -> Result<()> {
let sent = unsafe {
SendInput(inputs, std::mem::size_of::<INPUT>() as i32)
};
if sent as usize != inputs.len() {
anyhow::bail!("SendInput failed: sent {} of {} inputs", sent, inputs.len());
}
Ok(())
}
#[cfg(not(windows))]
pub fn move_to(&mut self, _x: i32, _y: i32) -> Result<()> {
anyhow::bail!("Mouse input only supported on Windows")
}
#[cfg(not(windows))]
pub fn button_down(&mut self, _button: MouseButton) -> Result<()> {
anyhow::bail!("Mouse input only supported on Windows")
}
#[cfg(not(windows))]
pub fn button_up(&mut self, _button: MouseButton) -> Result<()> {
anyhow::bail!("Mouse input only supported on Windows")
}
#[cfg(not(windows))]
pub fn scroll(&mut self, _delta_x: i32, _delta_y: i32) -> Result<()> {
anyhow::bail!("Mouse input only supported on Windows")
}
}

70
agent/src/main.rs Normal file
View File

@@ -0,0 +1,70 @@
//! GuruConnect Agent - Remote Desktop Agent for Windows
//!
//! Provides screen capture, input injection, and remote control capabilities.
mod capture;
mod config;
mod encoder;
mod input;
mod session;
mod transport;
pub mod proto {
include!(concat!(env!("OUT_DIR"), "/guruconnect.rs"));
}
use anyhow::Result;
use tracing::{info, error, Level};
use tracing_subscriber::FmtSubscriber;
#[tokio::main]
async fn main() -> Result<()> {
// Initialize logging
let subscriber = FmtSubscriber::builder()
.with_max_level(Level::INFO)
.with_target(true)
.with_thread_ids(true)
.init();
info!("GuruConnect Agent v{}", env!("CARGO_PKG_VERSION"));
// Load configuration
let config = config::Config::load()?;
info!("Loaded configuration for server: {}", config.server_url);
// Run the agent
if let Err(e) = run_agent(config).await {
error!("Agent error: {}", e);
return Err(e);
}
Ok(())
}
async fn run_agent(config: config::Config) -> Result<()> {
// Create session manager
let mut session = session::SessionManager::new(config.clone());
// Connect to server and run main loop
loop {
info!("Connecting to server...");
match session.connect().await {
Ok(_) => {
info!("Connected to server");
// Run session until disconnect
if let Err(e) = session.run().await {
error!("Session error: {}", e);
}
}
Err(e) => {
error!("Connection failed: {}", e);
}
}
// Wait before reconnecting
info!("Reconnecting in 5 seconds...");
tokio::time::sleep(tokio::time::Duration::from_secs(5)).await;
}
}

194
agent/src/session/mod.rs Normal file
View File

@@ -0,0 +1,194 @@
//! Session management for the agent
//!
//! Handles the lifecycle of a remote session including:
//! - Connection to server
//! - Authentication
//! - Frame capture and encoding loop
//! - Input event handling
use crate::capture::{self, Capturer, Display};
use crate::config::Config;
use crate::encoder::{self, Encoder};
use crate::input::InputController;
use crate::proto::{Message, message};
use crate::transport::WebSocketTransport;
use anyhow::Result;
use std::sync::Arc;
use std::time::{Duration, Instant};
use tokio::sync::mpsc;
/// Session manager handles the remote control session
pub struct SessionManager {
config: Config,
transport: Option<WebSocketTransport>,
state: SessionState,
}
#[derive(Debug, Clone, PartialEq)]
enum SessionState {
Disconnected,
Connecting,
Connected,
Active,
}
impl SessionManager {
/// Create a new session manager
pub fn new(config: Config) -> Self {
Self {
config,
transport: None,
state: SessionState::Disconnected,
}
}
/// Connect to the server
pub async fn connect(&mut self) -> Result<()> {
self.state = SessionState::Connecting;
let transport = WebSocketTransport::connect(
&self.config.server_url,
&self.config.api_key,
).await?;
self.transport = Some(transport);
self.state = SessionState::Connected;
Ok(())
}
/// Run the session main loop
pub async fn run(&mut self) -> Result<()> {
let transport = self.transport.as_mut()
.ok_or_else(|| anyhow::anyhow!("Not connected"))?;
self.state = SessionState::Active;
// Get primary display
let display = capture::primary_display()?;
tracing::info!("Using display: {} ({}x{})", display.name, display.width, display.height);
// Create capturer
let mut capturer = capture::create_capturer(
display.clone(),
self.config.capture.use_dxgi,
self.config.capture.gdi_fallback,
)?;
// Create encoder
let mut encoder = encoder::create_encoder(
&self.config.encoding.codec,
self.config.encoding.quality,
)?;
// Create input controller
let mut input = InputController::new()?;
// Calculate frame interval
let frame_interval = Duration::from_millis(1000 / self.config.capture.fps as u64);
let mut last_frame_time = Instant::now();
// Main loop
loop {
// Check for incoming messages (non-blocking)
while let Some(msg) = transport.try_recv()? {
self.handle_message(&mut input, msg)?;
}
// Capture and send frame if interval elapsed
if last_frame_time.elapsed() >= frame_interval {
last_frame_time = Instant::now();
if let Some(frame) = capturer.capture()? {
let encoded = encoder.encode(&frame)?;
// Skip empty frames (no changes)
if encoded.size > 0 {
let msg = Message {
payload: Some(message::Payload::VideoFrame(encoded.frame)),
};
transport.send(msg).await?;
}
}
}
// Small sleep to prevent busy loop
tokio::time::sleep(Duration::from_millis(1)).await;
// Check if still connected
if !transport.is_connected() {
tracing::warn!("Connection lost");
break;
}
}
self.state = SessionState::Disconnected;
Ok(())
}
/// Handle incoming message from server
fn handle_message(&mut self, input: &mut InputController, msg: Message) -> Result<()> {
match msg.payload {
Some(message::Payload::MouseEvent(mouse)) => {
// Handle mouse event
use crate::proto::MouseEventType;
use crate::input::MouseButton;
match MouseEventType::try_from(mouse.event_type).unwrap_or(MouseEventType::MouseMove) {
MouseEventType::MouseMove => {
input.mouse_move(mouse.x, mouse.y)?;
}
MouseEventType::MouseDown => {
input.mouse_move(mouse.x, mouse.y)?;
if let Some(ref buttons) = mouse.buttons {
if buttons.left { input.mouse_click(MouseButton::Left, true)?; }
if buttons.right { input.mouse_click(MouseButton::Right, true)?; }
if buttons.middle { input.mouse_click(MouseButton::Middle, true)?; }
}
}
MouseEventType::MouseUp => {
if let Some(ref buttons) = mouse.buttons {
if buttons.left { input.mouse_click(MouseButton::Left, false)?; }
if buttons.right { input.mouse_click(MouseButton::Right, false)?; }
if buttons.middle { input.mouse_click(MouseButton::Middle, false)?; }
}
}
MouseEventType::MouseWheel => {
input.mouse_scroll(mouse.wheel_delta_x, mouse.wheel_delta_y)?;
}
}
}
Some(message::Payload::KeyEvent(key)) => {
// Handle keyboard event
input.key_event(key.vk_code as u16, key.down)?;
}
Some(message::Payload::SpecialKey(special)) => {
use crate::proto::SpecialKey;
match SpecialKey::try_from(special.key).ok() {
Some(SpecialKey::CtrlAltDel) => {
input.send_ctrl_alt_del()?;
}
_ => {}
}
}
Some(message::Payload::Heartbeat(_)) => {
// Respond to heartbeat
// TODO: Send heartbeat ack
}
Some(message::Payload::Disconnect(disc)) => {
tracing::info!("Disconnect requested: {}", disc.reason);
return Err(anyhow::anyhow!("Disconnect: {}", disc.reason));
}
_ => {
// Ignore unknown messages
}
}
Ok(())
}
}

View File

@@ -0,0 +1,5 @@
//! WebSocket transport for agent-server communication
mod websocket;
pub use websocket::WebSocketTransport;

View File

@@ -0,0 +1,183 @@
//! WebSocket client transport
//!
//! Handles WebSocket connection to the GuruConnect server with:
//! - TLS encryption
//! - Automatic reconnection
//! - Protobuf message serialization
use crate::proto::Message;
use anyhow::{Context, Result};
use bytes::Bytes;
use futures_util::{SinkExt, StreamExt};
use prost::Message as ProstMessage;
use std::collections::VecDeque;
use std::sync::Arc;
use tokio::net::TcpStream;
use tokio::sync::Mutex;
use tokio_tungstenite::{
connect_async, tungstenite::protocol::Message as WsMessage, MaybeTlsStream, WebSocketStream,
};
type WsStream = WebSocketStream<MaybeTlsStream<TcpStream>>;
/// WebSocket transport for server communication
pub struct WebSocketTransport {
stream: Arc<Mutex<WsStream>>,
incoming: VecDeque<Message>,
connected: bool,
}
impl WebSocketTransport {
/// Connect to the server
pub async fn connect(url: &str, api_key: &str) -> Result<Self> {
// Append API key as query parameter
let url_with_auth = if url.contains('?') {
format!("{}&api_key={}", url, api_key)
} else {
format!("{}?api_key={}", url, api_key)
};
tracing::info!("Connecting to {}", url);
let (ws_stream, response) = connect_async(&url_with_auth)
.await
.context("Failed to connect to WebSocket server")?;
tracing::info!("Connected, status: {}", response.status());
Ok(Self {
stream: Arc::new(Mutex::new(ws_stream)),
incoming: VecDeque::new(),
connected: true,
})
}
/// Send a protobuf message
pub async fn send(&mut self, msg: Message) -> Result<()> {
let mut stream = self.stream.lock().await;
// Serialize to protobuf binary
let mut buf = Vec::with_capacity(msg.encoded_len());
msg.encode(&mut buf)?;
// Send as binary WebSocket message
stream
.send(WsMessage::Binary(buf.into()))
.await
.context("Failed to send message")?;
Ok(())
}
/// Try to receive a message (non-blocking)
pub fn try_recv(&mut self) -> Result<Option<Message>> {
// Return buffered message if available
if let Some(msg) = self.incoming.pop_front() {
return Ok(Some(msg));
}
// Try to receive more messages
let stream = self.stream.clone();
let result = tokio::task::block_in_place(|| {
tokio::runtime::Handle::current().block_on(async {
let mut stream = stream.lock().await;
// Use try_next for non-blocking receive
match tokio::time::timeout(
std::time::Duration::from_millis(1),
stream.next(),
)
.await
{
Ok(Some(Ok(ws_msg))) => Ok(Some(ws_msg)),
Ok(Some(Err(e))) => Err(anyhow::anyhow!("WebSocket error: {}", e)),
Ok(None) => {
// Connection closed
Ok(None)
}
Err(_) => {
// Timeout - no message available
Ok(None)
}
}
})
});
match result? {
Some(ws_msg) => {
if let Some(msg) = self.parse_message(ws_msg)? {
Ok(Some(msg))
} else {
Ok(None)
}
}
None => Ok(None),
}
}
/// Receive a message (blocking)
pub async fn recv(&mut self) -> Result<Option<Message>> {
// Return buffered message if available
if let Some(msg) = self.incoming.pop_front() {
return Ok(Some(msg));
}
let mut stream = self.stream.lock().await;
match stream.next().await {
Some(Ok(ws_msg)) => self.parse_message(ws_msg),
Some(Err(e)) => {
self.connected = false;
Err(anyhow::anyhow!("WebSocket error: {}", e))
}
None => {
self.connected = false;
Ok(None)
}
}
}
/// Parse a WebSocket message into a protobuf message
fn parse_message(&mut self, ws_msg: WsMessage) -> Result<Option<Message>> {
match ws_msg {
WsMessage::Binary(data) => {
let msg = Message::decode(Bytes::from(data))
.context("Failed to decode protobuf message")?;
Ok(Some(msg))
}
WsMessage::Ping(data) => {
// Pong is sent automatically by tungstenite
tracing::trace!("Received ping");
Ok(None)
}
WsMessage::Pong(_) => {
tracing::trace!("Received pong");
Ok(None)
}
WsMessage::Close(frame) => {
tracing::info!("Connection closed: {:?}", frame);
self.connected = false;
Ok(None)
}
WsMessage::Text(text) => {
// We expect binary protobuf, but log text messages
tracing::warn!("Received unexpected text message: {}", text);
Ok(None)
}
_ => Ok(None),
}
}
/// Check if connected
pub fn is_connected(&self) -> bool {
self.connected
}
/// Close the connection
pub async fn close(&mut self) -> Result<()> {
let mut stream = self.stream.lock().await;
stream.close(None).await?;
self.connected = false;
Ok(())
}
}

25
dashboard/package.json Normal file
View File

@@ -0,0 +1,25 @@
{
"name": "@guruconnect/dashboard",
"version": "0.1.0",
"description": "GuruConnect Remote Desktop Viewer Components",
"author": "AZ Computer Guru",
"license": "Proprietary",
"main": "src/components/index.ts",
"types": "src/components/index.ts",
"scripts": {
"typecheck": "tsc --noEmit",
"lint": "eslint src"
},
"peerDependencies": {
"react": "^18.0.0",
"react-dom": "^18.0.0"
},
"devDependencies": {
"@types/react": "^18.2.0",
"@types/react-dom": "^18.2.0",
"typescript": "^5.0.0"
},
"dependencies": {
"fzstd": "^0.1.1"
}
}

View File

@@ -0,0 +1,215 @@
/**
* RemoteViewer Component
*
* Canvas-based remote desktop viewer that connects to a GuruConnect
* agent via the relay server. Handles frame rendering and input capture.
*/
import React, { useRef, useEffect, useCallback, useState } from 'react';
import { useRemoteSession, createMouseEvent, createKeyEvent } from '../hooks/useRemoteSession';
import type { VideoFrame, ConnectionStatus, MouseEventType } from '../types/protocol';
interface RemoteViewerProps {
serverUrl: string;
sessionId: string;
className?: string;
onStatusChange?: (status: ConnectionStatus) => void;
autoConnect?: boolean;
showStatusBar?: boolean;
}
export const RemoteViewer: React.FC<RemoteViewerProps> = ({
serverUrl,
sessionId,
className = '',
onStatusChange,
autoConnect = true,
showStatusBar = true,
}) => {
const canvasRef = useRef<HTMLCanvasElement>(null);
const containerRef = useRef<HTMLDivElement>(null);
const ctxRef = useRef<CanvasRenderingContext2D | null>(null);
// Display dimensions from received frames
const [displaySize, setDisplaySize] = useState({ width: 1920, height: 1080 });
// Frame buffer for rendering
const frameBufferRef = useRef<ImageData | null>(null);
// Handle incoming video frames
const handleFrame = useCallback((frame: VideoFrame) => {
if (!frame.raw || !canvasRef.current) return;
const { width, height, data, compressed, isKeyframe } = frame.raw;
// Update display size if changed
if (width !== displaySize.width || height !== displaySize.height) {
setDisplaySize({ width, height });
}
// Get or create context
if (!ctxRef.current) {
ctxRef.current = canvasRef.current.getContext('2d', {
alpha: false,
desynchronized: true,
});
}
const ctx = ctxRef.current;
if (!ctx) return;
// For MVP, we assume raw BGRA frames
// In production, handle compressed frames with fzstd
let frameData = data;
// Create or reuse ImageData
if (!frameBufferRef.current ||
frameBufferRef.current.width !== width ||
frameBufferRef.current.height !== height) {
frameBufferRef.current = ctx.createImageData(width, height);
}
const imageData = frameBufferRef.current;
// Convert BGRA to RGBA for canvas
const pixels = imageData.data;
const len = Math.min(frameData.length, pixels.length);
for (let i = 0; i < len; i += 4) {
pixels[i] = frameData[i + 2]; // R <- B
pixels[i + 1] = frameData[i + 1]; // G <- G
pixels[i + 2] = frameData[i]; // B <- R
pixels[i + 3] = 255; // A (opaque)
}
// Draw to canvas
ctx.putImageData(imageData, 0, 0);
}, [displaySize]);
// Set up session
const { status, connect, disconnect, sendMouseEvent, sendKeyEvent } = useRemoteSession({
serverUrl,
sessionId,
onFrame: handleFrame,
onStatusChange,
});
// Auto-connect on mount
useEffect(() => {
if (autoConnect) {
connect();
}
return () => {
disconnect();
};
}, [autoConnect, connect, disconnect]);
// Update canvas size when display size changes
useEffect(() => {
if (canvasRef.current) {
canvasRef.current.width = displaySize.width;
canvasRef.current.height = displaySize.height;
// Reset context reference
ctxRef.current = null;
frameBufferRef.current = null;
}
}, [displaySize]);
// Get canvas rect for coordinate translation
const getCanvasRect = useCallback(() => {
return canvasRef.current?.getBoundingClientRect() ?? new DOMRect();
}, []);
// Mouse event handlers
const handleMouseMove = useCallback((e: React.MouseEvent<HTMLCanvasElement>) => {
const event = createMouseEvent(e, getCanvasRect(), displaySize.width, displaySize.height, 0);
sendMouseEvent(event);
}, [getCanvasRect, displaySize, sendMouseEvent]);
const handleMouseDown = useCallback((e: React.MouseEvent<HTMLCanvasElement>) => {
e.preventDefault();
const event = createMouseEvent(e, getCanvasRect(), displaySize.width, displaySize.height, 1);
sendMouseEvent(event);
}, [getCanvasRect, displaySize, sendMouseEvent]);
const handleMouseUp = useCallback((e: React.MouseEvent<HTMLCanvasElement>) => {
const event = createMouseEvent(e, getCanvasRect(), displaySize.width, displaySize.height, 2);
sendMouseEvent(event);
}, [getCanvasRect, displaySize, sendMouseEvent]);
const handleWheel = useCallback((e: React.WheelEvent<HTMLCanvasElement>) => {
e.preventDefault();
const baseEvent = createMouseEvent(e, getCanvasRect(), displaySize.width, displaySize.height, 3);
sendMouseEvent({
...baseEvent,
wheelDeltaX: Math.round(e.deltaX),
wheelDeltaY: Math.round(e.deltaY),
});
}, [getCanvasRect, displaySize, sendMouseEvent]);
const handleContextMenu = useCallback((e: React.MouseEvent<HTMLCanvasElement>) => {
e.preventDefault(); // Prevent browser context menu
}, []);
// Keyboard event handlers
const handleKeyDown = useCallback((e: React.KeyboardEvent<HTMLCanvasElement>) => {
e.preventDefault();
const event = createKeyEvent(e, true);
sendKeyEvent(event);
}, [sendKeyEvent]);
const handleKeyUp = useCallback((e: React.KeyboardEvent<HTMLCanvasElement>) => {
e.preventDefault();
const event = createKeyEvent(e, false);
sendKeyEvent(event);
}, [sendKeyEvent]);
return (
<div ref={containerRef} className={`remote-viewer ${className}`}>
<canvas
ref={canvasRef}
tabIndex={0}
onMouseMove={handleMouseMove}
onMouseDown={handleMouseDown}
onMouseUp={handleMouseUp}
onWheel={handleWheel}
onContextMenu={handleContextMenu}
onKeyDown={handleKeyDown}
onKeyUp={handleKeyUp}
style={{
width: '100%',
height: 'auto',
aspectRatio: `${displaySize.width} / ${displaySize.height}`,
cursor: 'none', // Hide cursor, remote cursor is shown in frame
outline: 'none',
backgroundColor: '#1a1a1a',
}}
/>
{showStatusBar && (
<div className="remote-viewer-status" style={{
display: 'flex',
justifyContent: 'space-between',
padding: '4px 8px',
backgroundColor: '#333',
color: '#fff',
fontSize: '12px',
fontFamily: 'monospace',
}}>
<span>
{status.connected ? (
<span style={{ color: '#4ade80' }}>Connected</span>
) : (
<span style={{ color: '#f87171' }}>Disconnected</span>
)}
</span>
<span>{displaySize.width}x{displaySize.height}</span>
{status.fps !== undefined && <span>{status.fps} FPS</span>}
{status.latencyMs !== undefined && <span>{status.latencyMs}ms</span>}
</div>
)}
</div>
);
};
export default RemoteViewer;

View File

@@ -0,0 +1,187 @@
/**
* Session Controls Component
*
* Toolbar for controlling the remote session (quality, displays, special keys)
*/
import React, { useState } from 'react';
import type { QualitySettings, Display } from '../types/protocol';
interface SessionControlsProps {
displays?: Display[];
currentDisplay?: number;
onDisplayChange?: (displayId: number) => void;
quality?: QualitySettings;
onQualityChange?: (settings: QualitySettings) => void;
onSpecialKey?: (key: 'ctrl-alt-del' | 'lock-screen' | 'print-screen') => void;
onDisconnect?: () => void;
}
export const SessionControls: React.FC<SessionControlsProps> = ({
displays = [],
currentDisplay = 0,
onDisplayChange,
quality,
onQualityChange,
onSpecialKey,
onDisconnect,
}) => {
const [showQuality, setShowQuality] = useState(false);
const handleQualityPreset = (preset: 'auto' | 'low' | 'balanced' | 'high') => {
onQualityChange?.({
preset,
codec: 'auto',
});
};
return (
<div className="session-controls" style={{
display: 'flex',
gap: '8px',
padding: '8px',
backgroundColor: '#222',
borderBottom: '1px solid #444',
}}>
{/* Display selector */}
{displays.length > 1 && (
<select
value={currentDisplay}
onChange={(e) => onDisplayChange?.(Number(e.target.value))}
style={{
padding: '4px 8px',
backgroundColor: '#333',
color: '#fff',
border: '1px solid #555',
borderRadius: '4px',
}}
>
{displays.map((d) => (
<option key={d.id} value={d.id}>
{d.name || `Display ${d.id + 1}`}
{d.isPrimary ? ' (Primary)' : ''}
</option>
))}
</select>
)}
{/* Quality dropdown */}
<div style={{ position: 'relative' }}>
<button
onClick={() => setShowQuality(!showQuality)}
style={{
padding: '4px 12px',
backgroundColor: '#333',
color: '#fff',
border: '1px solid #555',
borderRadius: '4px',
cursor: 'pointer',
}}
>
Quality: {quality?.preset || 'auto'}
</button>
{showQuality && (
<div style={{
position: 'absolute',
top: '100%',
left: 0,
marginTop: '4px',
backgroundColor: '#333',
border: '1px solid #555',
borderRadius: '4px',
zIndex: 100,
}}>
{(['auto', 'low', 'balanced', 'high'] as const).map((preset) => (
<button
key={preset}
onClick={() => {
handleQualityPreset(preset);
setShowQuality(false);
}}
style={{
display: 'block',
width: '100%',
padding: '8px 16px',
backgroundColor: quality?.preset === preset ? '#444' : 'transparent',
color: '#fff',
border: 'none',
textAlign: 'left',
cursor: 'pointer',
}}
>
{preset.charAt(0).toUpperCase() + preset.slice(1)}
</button>
))}
</div>
)}
</div>
{/* Special keys */}
<button
onClick={() => onSpecialKey?.('ctrl-alt-del')}
title="Send Ctrl+Alt+Delete"
style={{
padding: '4px 12px',
backgroundColor: '#333',
color: '#fff',
border: '1px solid #555',
borderRadius: '4px',
cursor: 'pointer',
}}
>
Ctrl+Alt+Del
</button>
<button
onClick={() => onSpecialKey?.('lock-screen')}
title="Lock Screen (Win+L)"
style={{
padding: '4px 12px',
backgroundColor: '#333',
color: '#fff',
border: '1px solid #555',
borderRadius: '4px',
cursor: 'pointer',
}}
>
Lock
</button>
<button
onClick={() => onSpecialKey?.('print-screen')}
title="Print Screen"
style={{
padding: '4px 12px',
backgroundColor: '#333',
color: '#fff',
border: '1px solid #555',
borderRadius: '4px',
cursor: 'pointer',
}}
>
PrtSc
</button>
{/* Spacer */}
<div style={{ flex: 1 }} />
{/* Disconnect */}
<button
onClick={onDisconnect}
style={{
padding: '4px 12px',
backgroundColor: '#dc2626',
color: '#fff',
border: 'none',
borderRadius: '4px',
cursor: 'pointer',
}}
>
Disconnect
</button>
</div>
);
};
export default SessionControls;

View File

@@ -0,0 +1,22 @@
/**
* GuruConnect Dashboard Components
*
* Export all components for use in GuruRMM dashboard
*/
export { RemoteViewer } from './RemoteViewer';
export { SessionControls } from './SessionControls';
// Re-export types
export type {
ConnectionStatus,
Display,
DisplayInfo,
QualitySettings,
VideoFrame,
MouseEvent as ProtoMouseEvent,
KeyEvent as ProtoKeyEvent,
} from '../types/protocol';
// Re-export hooks
export { useRemoteSession, createMouseEvent, createKeyEvent } from '../hooks/useRemoteSession';

View File

@@ -0,0 +1,239 @@
/**
* React hook for managing remote desktop session connection
*/
import { useState, useEffect, useCallback, useRef } from 'react';
import type { ConnectionStatus, VideoFrame, MouseEvent as ProtoMouseEvent, KeyEvent as ProtoKeyEvent, MouseEventType, KeyEventType, Modifiers } from '../types/protocol';
import { encodeMouseEvent, encodeKeyEvent, decodeVideoFrame } from '../lib/protobuf';
interface UseRemoteSessionOptions {
serverUrl: string;
sessionId: string;
onFrame?: (frame: VideoFrame) => void;
onStatusChange?: (status: ConnectionStatus) => void;
}
interface UseRemoteSessionReturn {
status: ConnectionStatus;
connect: () => void;
disconnect: () => void;
sendMouseEvent: (event: ProtoMouseEvent) => void;
sendKeyEvent: (event: ProtoKeyEvent) => void;
}
export function useRemoteSession(options: UseRemoteSessionOptions): UseRemoteSessionReturn {
const { serverUrl, sessionId, onFrame, onStatusChange } = options;
const [status, setStatus] = useState<ConnectionStatus>({
connected: false,
});
const wsRef = useRef<WebSocket | null>(null);
const reconnectTimeoutRef = useRef<number | null>(null);
const frameCountRef = useRef(0);
const lastFpsUpdateRef = useRef(Date.now());
// Update status and notify
const updateStatus = useCallback((newStatus: Partial<ConnectionStatus>) => {
setStatus(prev => {
const updated = { ...prev, ...newStatus };
onStatusChange?.(updated);
return updated;
});
}, [onStatusChange]);
// Calculate FPS
const updateFps = useCallback(() => {
const now = Date.now();
const elapsed = now - lastFpsUpdateRef.current;
if (elapsed >= 1000) {
const fps = Math.round((frameCountRef.current * 1000) / elapsed);
updateStatus({ fps });
frameCountRef.current = 0;
lastFpsUpdateRef.current = now;
}
}, [updateStatus]);
// Handle incoming WebSocket messages
const handleMessage = useCallback((event: MessageEvent) => {
if (event.data instanceof Blob) {
event.data.arrayBuffer().then(buffer => {
const data = new Uint8Array(buffer);
const frame = decodeVideoFrame(data);
if (frame) {
frameCountRef.current++;
updateFps();
onFrame?.(frame);
}
});
} else if (event.data instanceof ArrayBuffer) {
const data = new Uint8Array(event.data);
const frame = decodeVideoFrame(data);
if (frame) {
frameCountRef.current++;
updateFps();
onFrame?.(frame);
}
}
}, [onFrame, updateFps]);
// Connect to server
const connect = useCallback(() => {
if (wsRef.current?.readyState === WebSocket.OPEN) {
return;
}
// Clear any pending reconnect
if (reconnectTimeoutRef.current) {
window.clearTimeout(reconnectTimeoutRef.current);
reconnectTimeoutRef.current = null;
}
const wsUrl = `${serverUrl}/ws/viewer?session_id=${encodeURIComponent(sessionId)}`;
const ws = new WebSocket(wsUrl);
ws.binaryType = 'arraybuffer';
ws.onopen = () => {
updateStatus({
connected: true,
sessionId,
});
};
ws.onmessage = handleMessage;
ws.onclose = (event) => {
updateStatus({
connected: false,
latencyMs: undefined,
fps: undefined,
});
// Auto-reconnect after 2 seconds
if (!event.wasClean) {
reconnectTimeoutRef.current = window.setTimeout(() => {
connect();
}, 2000);
}
};
ws.onerror = () => {
updateStatus({ connected: false });
};
wsRef.current = ws;
}, [serverUrl, sessionId, handleMessage, updateStatus]);
// Disconnect from server
const disconnect = useCallback(() => {
if (reconnectTimeoutRef.current) {
window.clearTimeout(reconnectTimeoutRef.current);
reconnectTimeoutRef.current = null;
}
if (wsRef.current) {
wsRef.current.close(1000, 'User disconnected');
wsRef.current = null;
}
updateStatus({
connected: false,
sessionId: undefined,
latencyMs: undefined,
fps: undefined,
});
}, [updateStatus]);
// Send mouse event
const sendMouseEvent = useCallback((event: ProtoMouseEvent) => {
if (wsRef.current?.readyState === WebSocket.OPEN) {
const data = encodeMouseEvent(event);
wsRef.current.send(data);
}
}, []);
// Send key event
const sendKeyEvent = useCallback((event: ProtoKeyEvent) => {
if (wsRef.current?.readyState === WebSocket.OPEN) {
const data = encodeKeyEvent(event);
wsRef.current.send(data);
}
}, []);
// Cleanup on unmount
useEffect(() => {
return () => {
disconnect();
};
}, [disconnect]);
return {
status,
connect,
disconnect,
sendMouseEvent,
sendKeyEvent,
};
}
/**
* Helper to create mouse event from DOM mouse event
*/
export function createMouseEvent(
domEvent: React.MouseEvent<HTMLElement>,
canvasRect: DOMRect,
displayWidth: number,
displayHeight: number,
eventType: MouseEventType
): ProtoMouseEvent {
// Calculate position relative to canvas and scale to display coordinates
const scaleX = displayWidth / canvasRect.width;
const scaleY = displayHeight / canvasRect.height;
const x = Math.round((domEvent.clientX - canvasRect.left) * scaleX);
const y = Math.round((domEvent.clientY - canvasRect.top) * scaleY);
return {
x,
y,
buttons: {
left: (domEvent.buttons & 1) !== 0,
right: (domEvent.buttons & 2) !== 0,
middle: (domEvent.buttons & 4) !== 0,
x1: (domEvent.buttons & 8) !== 0,
x2: (domEvent.buttons & 16) !== 0,
},
wheelDeltaX: 0,
wheelDeltaY: 0,
eventType,
};
}
/**
* Helper to create key event from DOM keyboard event
*/
export function createKeyEvent(
domEvent: React.KeyboardEvent<HTMLElement>,
down: boolean
): ProtoKeyEvent {
const modifiers: Modifiers = {
ctrl: domEvent.ctrlKey,
alt: domEvent.altKey,
shift: domEvent.shiftKey,
meta: domEvent.metaKey,
capsLock: domEvent.getModifierState('CapsLock'),
numLock: domEvent.getModifierState('NumLock'),
};
// Use key code for special keys, unicode for regular characters
const isCharacter = domEvent.key.length === 1;
return {
down,
keyType: isCharacter ? 2 : 0, // KEY_UNICODE or KEY_VK
vkCode: domEvent.keyCode,
scanCode: 0, // Not available in browser
unicode: isCharacter ? domEvent.key : undefined,
modifiers,
};
}

View File

@@ -0,0 +1,162 @@
/**
* Minimal protobuf encoder/decoder for GuruConnect messages
*
* For MVP, we use a simplified binary format. In production,
* this would use a proper protobuf library like protobufjs.
*/
import type { MouseEvent, KeyEvent, MouseEventType, KeyEventType, VideoFrame, RawFrame } from '../types/protocol';
// Message type identifiers (matching proto field numbers)
const MSG_VIDEO_FRAME = 10;
const MSG_MOUSE_EVENT = 20;
const MSG_KEY_EVENT = 21;
/**
* Encode a mouse event to binary format
*/
export function encodeMouseEvent(event: MouseEvent): Uint8Array {
const buffer = new ArrayBuffer(32);
const view = new DataView(buffer);
// Message type
view.setUint8(0, MSG_MOUSE_EVENT);
// Event type
view.setUint8(1, event.eventType);
// Coordinates (scaled to 16-bit for efficiency)
view.setInt16(2, event.x, true);
view.setInt16(4, event.y, true);
// Buttons bitmask
let buttons = 0;
if (event.buttons.left) buttons |= 1;
if (event.buttons.right) buttons |= 2;
if (event.buttons.middle) buttons |= 4;
if (event.buttons.x1) buttons |= 8;
if (event.buttons.x2) buttons |= 16;
view.setUint8(6, buttons);
// Wheel deltas
view.setInt16(7, event.wheelDeltaX, true);
view.setInt16(9, event.wheelDeltaY, true);
return new Uint8Array(buffer, 0, 11);
}
/**
* Encode a key event to binary format
*/
export function encodeKeyEvent(event: KeyEvent): Uint8Array {
const buffer = new ArrayBuffer(32);
const view = new DataView(buffer);
// Message type
view.setUint8(0, MSG_KEY_EVENT);
// Key down/up
view.setUint8(1, event.down ? 1 : 0);
// Key type
view.setUint8(2, event.keyType);
// Virtual key code
view.setUint16(3, event.vkCode, true);
// Scan code
view.setUint16(5, event.scanCode, true);
// Modifiers bitmask
let mods = 0;
if (event.modifiers.ctrl) mods |= 1;
if (event.modifiers.alt) mods |= 2;
if (event.modifiers.shift) mods |= 4;
if (event.modifiers.meta) mods |= 8;
if (event.modifiers.capsLock) mods |= 16;
if (event.modifiers.numLock) mods |= 32;
view.setUint8(7, mods);
// Unicode character (if present)
if (event.unicode && event.unicode.length > 0) {
const charCode = event.unicode.charCodeAt(0);
view.setUint16(8, charCode, true);
return new Uint8Array(buffer, 0, 10);
}
return new Uint8Array(buffer, 0, 8);
}
/**
* Decode a video frame from binary format
*/
export function decodeVideoFrame(data: Uint8Array): VideoFrame | null {
if (data.length < 2) return null;
const view = new DataView(data.buffer, data.byteOffset, data.byteLength);
const msgType = view.getUint8(0);
if (msgType !== MSG_VIDEO_FRAME) return null;
const encoding = view.getUint8(1);
const displayId = view.getUint8(2);
const sequence = view.getUint32(3, true);
const timestamp = Number(view.getBigInt64(7, true));
// Frame dimensions
const width = view.getUint16(15, true);
const height = view.getUint16(17, true);
// Compressed flag
const compressed = view.getUint8(19) === 1;
// Is keyframe
const isKeyframe = view.getUint8(20) === 1;
// Frame data starts at offset 21
const frameData = data.slice(21);
const encodingStr = ['raw', 'vp9', 'h264', 'h265'][encoding] as 'raw' | 'vp9' | 'h264' | 'h265';
if (encodingStr === 'raw') {
return {
timestamp,
displayId,
sequence,
encoding: 'raw',
raw: {
width,
height,
data: frameData,
compressed,
dirtyRects: [], // TODO: Parse dirty rects
isKeyframe,
},
};
}
return {
timestamp,
displayId,
sequence,
encoding: encodingStr,
encoded: {
data: frameData,
keyframe: isKeyframe,
pts: timestamp,
dts: timestamp,
},
};
}
/**
* Simple zstd decompression placeholder
* In production, use a proper zstd library like fzstd
*/
export async function decompressZstd(data: Uint8Array): Promise<Uint8Array> {
// For MVP, assume uncompressed frames or use fzstd library
// This is a placeholder - actual implementation would use:
// import { decompress } from 'fzstd';
// return decompress(data);
return data;
}

View File

@@ -0,0 +1,135 @@
/**
* TypeScript types matching guruconnect.proto definitions
* These are used for WebSocket message handling in the viewer
*/
export enum SessionType {
SCREEN_CONTROL = 0,
VIEW_ONLY = 1,
BACKSTAGE = 2,
FILE_TRANSFER = 3,
}
export interface SessionRequest {
agentId: string;
sessionToken: string;
sessionType: SessionType;
clientVersion: string;
}
export interface SessionResponse {
success: boolean;
sessionId: string;
error?: string;
displayInfo?: DisplayInfo;
}
export interface DisplayInfo {
displays: Display[];
primaryDisplay: number;
}
export interface Display {
id: number;
name: string;
x: number;
y: number;
width: number;
height: number;
isPrimary: boolean;
}
export interface DirtyRect {
x: number;
y: number;
width: number;
height: number;
}
export interface RawFrame {
width: number;
height: number;
data: Uint8Array;
compressed: boolean;
dirtyRects: DirtyRect[];
isKeyframe: boolean;
}
export interface EncodedFrame {
data: Uint8Array;
keyframe: boolean;
pts: number;
dts: number;
}
export interface VideoFrame {
timestamp: number;
displayId: number;
sequence: number;
encoding: 'raw' | 'vp9' | 'h264' | 'h265';
raw?: RawFrame;
encoded?: EncodedFrame;
}
export enum MouseEventType {
MOUSE_MOVE = 0,
MOUSE_DOWN = 1,
MOUSE_UP = 2,
MOUSE_WHEEL = 3,
}
export interface MouseButtons {
left: boolean;
right: boolean;
middle: boolean;
x1: boolean;
x2: boolean;
}
export interface MouseEvent {
x: number;
y: number;
buttons: MouseButtons;
wheelDeltaX: number;
wheelDeltaY: number;
eventType: MouseEventType;
}
export enum KeyEventType {
KEY_VK = 0,
KEY_SCAN = 1,
KEY_UNICODE = 2,
}
export interface Modifiers {
ctrl: boolean;
alt: boolean;
shift: boolean;
meta: boolean;
capsLock: boolean;
numLock: boolean;
}
export interface KeyEvent {
down: boolean;
keyType: KeyEventType;
vkCode: number;
scanCode: number;
unicode?: string;
modifiers: Modifiers;
}
export interface QualitySettings {
preset: 'auto' | 'low' | 'balanced' | 'high';
customFps?: number;
customBitrate?: number;
codec: 'auto' | 'raw' | 'vp9' | 'h264' | 'h265';
}
export interface ConnectionStatus {
connected: boolean;
sessionId?: string;
latencyMs?: number;
fps?: number;
bitrateKbps?: number;
}

21
dashboard/tsconfig.json Normal file
View File

@@ -0,0 +1,21 @@
{
"compilerOptions": {
"target": "ES2020",
"lib": ["ES2020", "DOM", "DOM.Iterable"],
"module": "ESNext",
"moduleResolution": "bundler",
"jsx": "react-jsx",
"strict": true,
"noEmit": true,
"esModuleInterop": true,
"skipLibCheck": true,
"forceConsistentCasingInFileNames": true,
"resolveJsonModule": true,
"isolatedModules": true,
"noUnusedLocals": true,
"noUnusedParameters": true,
"noFallthroughCasesInSwitch": true
},
"include": ["src"],
"exclude": ["node_modules"]
}

286
proto/guruconnect.proto Normal file
View File

@@ -0,0 +1,286 @@
syntax = "proto3";
package guruconnect;
// ============================================================================
// Session Management
// ============================================================================
message SessionRequest {
string agent_id = 1;
string session_token = 2;
SessionType session_type = 3;
string client_version = 4;
}
message SessionResponse {
bool success = 1;
string session_id = 2;
string error = 3;
DisplayInfo display_info = 4;
}
enum SessionType {
SCREEN_CONTROL = 0;
VIEW_ONLY = 1;
BACKSTAGE = 2;
FILE_TRANSFER = 3;
}
// ============================================================================
// Display Information
// ============================================================================
message DisplayInfo {
repeated Display displays = 1;
int32 primary_display = 2;
}
message Display {
int32 id = 1;
string name = 2;
int32 x = 3;
int32 y = 4;
int32 width = 5;
int32 height = 6;
bool is_primary = 7;
}
message SwitchDisplay {
int32 display_id = 1;
}
// ============================================================================
// Video Frames
// ============================================================================
message VideoFrame {
int64 timestamp = 1;
int32 display_id = 2;
int32 sequence = 3;
oneof encoding {
RawFrame raw = 10;
EncodedFrame vp9 = 11;
EncodedFrame h264 = 12;
EncodedFrame h265 = 13;
}
}
message RawFrame {
int32 width = 1;
int32 height = 2;
bytes data = 3; // Zstd compressed BGRA
bool compressed = 4;
repeated DirtyRect dirty_rects = 5;
bool is_keyframe = 6; // Full frame vs incremental
}
message DirtyRect {
int32 x = 1;
int32 y = 2;
int32 width = 3;
int32 height = 4;
}
message EncodedFrame {
bytes data = 1;
bool keyframe = 2;
int64 pts = 3;
int64 dts = 4;
}
message VideoAck {
int32 sequence = 1;
int64 timestamp = 2;
}
// ============================================================================
// Cursor
// ============================================================================
message CursorShape {
uint64 id = 1;
int32 hotspot_x = 2;
int32 hotspot_y = 3;
int32 width = 4;
int32 height = 5;
bytes data = 6; // BGRA bitmap
}
message CursorPosition {
int32 x = 1;
int32 y = 2;
bool visible = 3;
}
// ============================================================================
// Input Events
// ============================================================================
message MouseEvent {
int32 x = 1;
int32 y = 2;
MouseButtons buttons = 3;
int32 wheel_delta_x = 4;
int32 wheel_delta_y = 5;
MouseEventType event_type = 6;
}
enum MouseEventType {
MOUSE_MOVE = 0;
MOUSE_DOWN = 1;
MOUSE_UP = 2;
MOUSE_WHEEL = 3;
}
message MouseButtons {
bool left = 1;
bool right = 2;
bool middle = 3;
bool x1 = 4;
bool x2 = 5;
}
message KeyEvent {
bool down = 1; // true = key down, false = key up
KeyEventType key_type = 2;
uint32 vk_code = 3; // Virtual key code (Windows VK_*)
uint32 scan_code = 4; // Hardware scan code
string unicode = 5; // Unicode character (for text input)
Modifiers modifiers = 6;
}
enum KeyEventType {
KEY_VK = 0; // Virtual key code
KEY_SCAN = 1; // Scan code
KEY_UNICODE = 2; // Unicode character
}
message Modifiers {
bool ctrl = 1;
bool alt = 2;
bool shift = 3;
bool meta = 4; // Windows key
bool caps_lock = 5;
bool num_lock = 6;
}
message SpecialKeyEvent {
SpecialKey key = 1;
}
enum SpecialKey {
CTRL_ALT_DEL = 0;
LOCK_SCREEN = 1;
PRINT_SCREEN = 2;
}
// ============================================================================
// Clipboard
// ============================================================================
message ClipboardData {
ClipboardFormat format = 1;
bytes data = 2;
string mime_type = 3;
}
enum ClipboardFormat {
CLIPBOARD_TEXT = 0;
CLIPBOARD_HTML = 1;
CLIPBOARD_RTF = 2;
CLIPBOARD_IMAGE = 3;
CLIPBOARD_FILES = 4;
}
message ClipboardRequest {
// Request current clipboard content
}
// ============================================================================
// Quality Control
// ============================================================================
message QualitySettings {
QualityPreset preset = 1;
int32 custom_fps = 2; // 1-60
int32 custom_bitrate = 3; // kbps
CodecPreference codec = 4;
}
enum QualityPreset {
QUALITY_AUTO = 0;
QUALITY_LOW = 1; // Low bandwidth
QUALITY_BALANCED = 2;
QUALITY_HIGH = 3; // Best quality
}
enum CodecPreference {
CODEC_AUTO = 0;
CODEC_RAW = 1; // Raw + Zstd (LAN)
CODEC_VP9 = 2;
CODEC_H264 = 3;
CODEC_H265 = 4;
}
message LatencyReport {
int64 rtt_ms = 1;
int32 fps = 2;
int32 bitrate_kbps = 3;
}
// ============================================================================
// Control Messages
// ============================================================================
message Heartbeat {
int64 timestamp = 1;
}
message HeartbeatAck {
int64 client_timestamp = 1;
int64 server_timestamp = 2;
}
message Disconnect {
string reason = 1;
}
// ============================================================================
// Top-Level Message Wrapper
// ============================================================================
message Message {
oneof payload {
// Session
SessionRequest session_request = 1;
SessionResponse session_response = 2;
// Video
VideoFrame video_frame = 10;
VideoAck video_ack = 11;
SwitchDisplay switch_display = 12;
// Cursor
CursorShape cursor_shape = 15;
CursorPosition cursor_position = 16;
// Input
MouseEvent mouse_event = 20;
KeyEvent key_event = 21;
SpecialKeyEvent special_key = 22;
// Clipboard
ClipboardData clipboard_data = 30;
ClipboardRequest clipboard_request = 31;
// Quality
QualitySettings quality_settings = 40;
LatencyReport latency_report = 41;
// Control
Heartbeat heartbeat = 50;
HeartbeatAck heartbeat_ack = 51;
Disconnect disconnect = 52;
}
}

62
server/Cargo.toml Normal file
View File

@@ -0,0 +1,62 @@
[package]
name = "guruconnect-server"
version = "0.1.0"
edition = "2021"
authors = ["AZ Computer Guru"]
description = "GuruConnect Remote Desktop Relay Server"
[dependencies]
# Async runtime
tokio = { version = "1", features = ["full", "sync", "time", "rt-multi-thread", "macros"] }
# Web framework
axum = { version = "0.7", features = ["ws", "macros"] }
tower = "0.5"
tower-http = { version = "0.6", features = ["cors", "trace", "compression-gzip"] }
# WebSocket
futures-util = "0.3"
# Database
sqlx = { version = "0.8", features = ["runtime-tokio", "postgres", "uuid", "chrono", "json"] }
# Protocol (protobuf)
prost = "0.13"
prost-types = "0.13"
bytes = "1"
# Serialization
serde = { version = "1", features = ["derive"] }
serde_json = "1"
# Logging
tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
# Error handling
anyhow = "1"
thiserror = "1"
# Configuration
toml = "0.8"
# Auth
jsonwebtoken = "9"
argon2 = "0.5"
# Crypto
ring = "0.17"
# UUID
uuid = { version = "1", features = ["v4", "serde"] }
# Time
chrono = { version = "0.4", features = ["serde"] }
[build-dependencies]
prost-build = "0.13"
[profile.release]
lto = true
codegen-units = 1
strip = true

11
server/build.rs Normal file
View File

@@ -0,0 +1,11 @@
use std::io::Result;
fn main() -> Result<()> {
// Compile protobuf definitions
prost_build::compile_protos(&["../proto/guruconnect.proto"], &["../proto/"])?;
// Rerun if proto changes
println!("cargo:rerun-if-changed=../proto/guruconnect.proto");
Ok(())
}

54
server/src/api/mod.rs Normal file
View File

@@ -0,0 +1,54 @@
//! REST API endpoints
use axum::{
extract::{Path, State},
Json,
};
use serde::Serialize;
use uuid::Uuid;
use crate::session::SessionManager;
/// Session info returned by API
#[derive(Debug, Serialize)]
pub struct SessionInfo {
pub id: String,
pub agent_id: String,
pub agent_name: String,
pub started_at: String,
pub viewer_count: usize,
}
impl From<crate::session::Session> for SessionInfo {
fn from(s: crate::session::Session) -> Self {
Self {
id: s.id.to_string(),
agent_id: s.agent_id,
agent_name: s.agent_name,
started_at: s.started_at.to_rfc3339(),
viewer_count: s.viewer_count,
}
}
}
/// List all active sessions
pub async fn list_sessions(
State(sessions): State<SessionManager>,
) -> Json<Vec<SessionInfo>> {
let sessions = sessions.list_sessions().await;
Json(sessions.into_iter().map(SessionInfo::from).collect())
}
/// Get a specific session by ID
pub async fn get_session(
State(sessions): State<SessionManager>,
Path(id): Path<String>,
) -> Result<Json<SessionInfo>, (axum::http::StatusCode, &'static str)> {
let session_id = Uuid::parse_str(&id)
.map_err(|_| (axum::http::StatusCode::BAD_REQUEST, "Invalid session ID"))?;
let session = sessions.get_session(session_id).await
.ok_or((axum::http::StatusCode::NOT_FOUND, "Session not found"))?;
Ok(Json(SessionInfo::from(session)))
}

61
server/src/auth/mod.rs Normal file
View File

@@ -0,0 +1,61 @@
//! Authentication module
//!
//! Handles JWT validation for dashboard users and API key
//! validation for agents.
use axum::{
extract::FromRequestParts,
http::{request::Parts, StatusCode},
};
/// Authenticated user from JWT
#[derive(Debug, Clone)]
pub struct AuthenticatedUser {
pub user_id: String,
pub email: String,
pub roles: Vec<String>,
}
/// Authenticated agent from API key
#[derive(Debug, Clone)]
pub struct AuthenticatedAgent {
pub agent_id: String,
pub org_id: String,
}
/// Extract authenticated user from request (placeholder for MVP)
#[axum::async_trait]
impl<S> FromRequestParts<S> for AuthenticatedUser
where
S: Send + Sync,
{
type Rejection = (StatusCode, &'static str);
async fn from_request_parts(parts: &mut Parts, _state: &S) -> Result<Self, Self::Rejection> {
// TODO: Implement JWT validation
// For MVP, accept any request
// Look for Authorization header
let _auth_header = parts
.headers
.get("Authorization")
.and_then(|v| v.to_str().ok());
// Placeholder - in production, validate JWT
Ok(AuthenticatedUser {
user_id: "mvp-user".to_string(),
email: "mvp@example.com".to_string(),
roles: vec!["admin".to_string()],
})
}
}
/// Validate an agent API key (placeholder for MVP)
pub fn validate_agent_key(_api_key: &str) -> Option<AuthenticatedAgent> {
// TODO: Implement actual API key validation
// For MVP, accept any key
Some(AuthenticatedAgent {
agent_id: "mvp-agent".to_string(),
org_id: "mvp-org".to_string(),
})
}

45
server/src/config.rs Normal file
View File

@@ -0,0 +1,45 @@
//! Server configuration
use anyhow::Result;
use serde::Deserialize;
use std::env;
#[derive(Debug, Clone, Deserialize)]
pub struct Config {
/// Address to listen on (e.g., "0.0.0.0:8080")
pub listen_addr: String,
/// Database URL (optional for MVP)
pub database_url: Option<String>,
/// JWT secret for authentication
pub jwt_secret: Option<String>,
/// Enable debug logging
pub debug: bool,
}
impl Config {
/// Load configuration from environment variables
pub fn load() -> Result<Self> {
Ok(Self {
listen_addr: env::var("LISTEN_ADDR").unwrap_or_else(|_| "0.0.0.0:8080".to_string()),
database_url: env::var("DATABASE_URL").ok(),
jwt_secret: env::var("JWT_SECRET").ok(),
debug: env::var("DEBUG")
.map(|v| v == "1" || v.to_lowercase() == "true")
.unwrap_or(false),
})
}
}
impl Default for Config {
fn default() -> Self {
Self {
listen_addr: "0.0.0.0:8080".to_string(),
database_url: None,
jwt_secret: None,
debug: false,
}
}
}

45
server/src/db/mod.rs Normal file
View File

@@ -0,0 +1,45 @@
//! Database module
//!
//! Handles session logging and persistence.
//! Optional for MVP - sessions are kept in memory only.
use anyhow::Result;
/// Database connection pool (placeholder)
#[derive(Clone)]
pub struct Database {
// TODO: Add sqlx pool when PostgreSQL is needed
_placeholder: (),
}
impl Database {
/// Initialize database connection
pub async fn init(_database_url: &str) -> Result<Self> {
// TODO: Initialize PostgreSQL connection pool
Ok(Self { _placeholder: () })
}
}
/// Session event for audit logging
#[derive(Debug)]
pub struct SessionEvent {
pub session_id: String,
pub event_type: SessionEventType,
pub details: Option<String>,
}
#[derive(Debug)]
pub enum SessionEventType {
Started,
ViewerJoined,
ViewerLeft,
Ended,
}
impl Database {
/// Log a session event (placeholder)
pub async fn log_session_event(&self, _event: SessionEvent) -> Result<()> {
// TODO: Insert into connect_session_events table
Ok(())
}
}

82
server/src/main.rs Normal file
View File

@@ -0,0 +1,82 @@
//! GuruConnect Server - WebSocket Relay Server
//!
//! Handles connections from both agents and dashboard viewers,
//! relaying video frames and input events between them.
mod config;
mod relay;
mod session;
mod auth;
mod api;
mod db;
pub mod proto {
include!(concat!(env!("OUT_DIR"), "/guruconnect.rs"));
}
use anyhow::Result;
use axum::{
Router,
routing::get,
};
use std::net::SocketAddr;
use tower_http::cors::{Any, CorsLayer};
use tower_http::trace::TraceLayer;
use tracing::{info, Level};
use tracing_subscriber::FmtSubscriber;
#[tokio::main]
async fn main() -> Result<()> {
// Initialize logging
let _subscriber = FmtSubscriber::builder()
.with_max_level(Level::INFO)
.with_target(true)
.init();
info!("GuruConnect Server v{}", env!("CARGO_PKG_VERSION"));
// Load configuration
let config = config::Config::load()?;
info!("Loaded configuration, listening on {}", config.listen_addr);
// Initialize database connection (optional for MVP)
// let db = db::init(&config.database_url).await?;
// Create session manager
let sessions = session::SessionManager::new();
// Build router
let app = Router::new()
// Health check
.route("/health", get(health))
// WebSocket endpoints
.route("/ws/agent", get(relay::agent_ws_handler))
.route("/ws/viewer", get(relay::viewer_ws_handler))
// REST API
.route("/api/sessions", get(api::list_sessions))
.route("/api/sessions/:id", get(api::get_session))
// State
.with_state(sessions)
// Middleware
.layer(TraceLayer::new_for_http())
.layer(
CorsLayer::new()
.allow_origin(Any)
.allow_methods(Any)
.allow_headers(Any),
);
// Start server
let addr: SocketAddr = config.listen_addr.parse()?;
let listener = tokio::net::TcpListener::bind(addr).await?;
info!("Server listening on {}", addr);
axum::serve(listener, app).await?;
Ok(())
}
async fn health() -> &'static str {
"OK"
}

194
server/src/relay/mod.rs Normal file
View File

@@ -0,0 +1,194 @@
//! WebSocket relay handlers
//!
//! Handles WebSocket connections from agents and viewers,
//! relaying video frames and input events between them.
use axum::{
extract::{
ws::{Message, WebSocket, WebSocketUpgrade},
Query, State,
},
response::IntoResponse,
};
use futures_util::{SinkExt, StreamExt};
use prost::Message as ProstMessage;
use serde::Deserialize;
use tracing::{error, info, warn};
use crate::proto;
use crate::session::SessionManager;
#[derive(Debug, Deserialize)]
pub struct AgentParams {
agent_id: String,
#[serde(default)]
agent_name: Option<String>,
}
#[derive(Debug, Deserialize)]
pub struct ViewerParams {
session_id: String,
}
/// WebSocket handler for agent connections
pub async fn agent_ws_handler(
ws: WebSocketUpgrade,
State(sessions): State<SessionManager>,
Query(params): Query<AgentParams>,
) -> impl IntoResponse {
let agent_id = params.agent_id;
let agent_name = params.agent_name.unwrap_or_else(|| agent_id.clone());
ws.on_upgrade(move |socket| handle_agent_connection(socket, sessions, agent_id, agent_name))
}
/// WebSocket handler for viewer connections
pub async fn viewer_ws_handler(
ws: WebSocketUpgrade,
State(sessions): State<SessionManager>,
Query(params): Query<ViewerParams>,
) -> impl IntoResponse {
let session_id = params.session_id;
ws.on_upgrade(move |socket| handle_viewer_connection(socket, sessions, session_id))
}
/// Handle an agent WebSocket connection
async fn handle_agent_connection(
socket: WebSocket,
sessions: SessionManager,
agent_id: String,
agent_name: String,
) {
info!("Agent connected: {} ({})", agent_name, agent_id);
// Register the agent and get channels
let (session_id, frame_tx, mut input_rx) = sessions.register_agent(agent_id.clone(), agent_name.clone()).await;
info!("Session created: {}", session_id);
let (mut ws_sender, mut ws_receiver) = socket.split();
// Task to forward input events from viewers to agent
let input_forward = tokio::spawn(async move {
while let Some(input_data) = input_rx.recv().await {
if ws_sender.send(Message::Binary(input_data.into())).await.is_err() {
break;
}
}
});
// Main loop: receive frames from agent and broadcast to viewers
while let Some(msg) = ws_receiver.next().await {
match msg {
Ok(Message::Binary(data)) => {
// Try to decode as protobuf message
match proto::Message::decode(data.as_ref()) {
Ok(proto_msg) => {
if let Some(proto::message::Payload::VideoFrame(_)) = &proto_msg.payload {
// Broadcast frame to all viewers
let _ = frame_tx.send(data.to_vec());
}
}
Err(e) => {
warn!("Failed to decode agent message: {}", e);
}
}
}
Ok(Message::Close(_)) => {
info!("Agent disconnected: {}", agent_id);
break;
}
Ok(Message::Ping(data)) => {
// Pong is handled automatically by axum
let _ = data;
}
Ok(_) => {}
Err(e) => {
error!("WebSocket error from agent {}: {}", agent_id, e);
break;
}
}
}
// Cleanup
input_forward.abort();
sessions.remove_session(session_id).await;
info!("Session {} ended", session_id);
}
/// Handle a viewer WebSocket connection
async fn handle_viewer_connection(
socket: WebSocket,
sessions: SessionManager,
session_id_str: String,
) {
// Parse session ID
let session_id = match uuid::Uuid::parse_str(&session_id_str) {
Ok(id) => id,
Err(_) => {
warn!("Invalid session ID: {}", session_id_str);
return;
}
};
// Join the session
let (mut frame_rx, input_tx) = match sessions.join_session(session_id).await {
Some(channels) => channels,
None => {
warn!("Session not found: {}", session_id);
return;
}
};
info!("Viewer joined session: {}", session_id);
let (mut ws_sender, mut ws_receiver) = socket.split();
// Task to forward frames from agent to this viewer
let frame_forward = tokio::spawn(async move {
while let Ok(frame_data) = frame_rx.recv().await {
if ws_sender.send(Message::Binary(frame_data.into())).await.is_err() {
break;
}
}
});
// Main loop: receive input from viewer and forward to agent
while let Some(msg) = ws_receiver.next().await {
match msg {
Ok(Message::Binary(data)) => {
// Try to decode as protobuf message
match proto::Message::decode(data.as_ref()) {
Ok(proto_msg) => {
match &proto_msg.payload {
Some(proto::message::Payload::MouseEvent(_)) |
Some(proto::message::Payload::KeyEvent(_)) => {
// Forward input to agent
let _ = input_tx.send(data.to_vec()).await;
}
_ => {}
}
}
Err(e) => {
warn!("Failed to decode viewer message: {}", e);
}
}
}
Ok(Message::Close(_)) => {
info!("Viewer disconnected from session: {}", session_id);
break;
}
Ok(_) => {}
Err(e) => {
error!("WebSocket error from viewer: {}", e);
break;
}
}
}
// Cleanup
frame_forward.abort();
sessions.leave_session(session_id).await;
info!("Viewer left session: {}", session_id);
}

148
server/src/session/mod.rs Normal file
View File

@@ -0,0 +1,148 @@
//! Session management for GuruConnect
//!
//! Manages active remote desktop sessions, tracking which agents
//! are connected and which viewers are watching them.
use std::collections::HashMap;
use std::sync::Arc;
use tokio::sync::{broadcast, RwLock};
use uuid::Uuid;
/// Unique identifier for a session
pub type SessionId = Uuid;
/// Unique identifier for an agent
pub type AgentId = String;
/// Session state
#[derive(Debug, Clone)]
pub struct Session {
pub id: SessionId,
pub agent_id: AgentId,
pub agent_name: String,
pub started_at: chrono::DateTime<chrono::Utc>,
pub viewer_count: usize,
}
/// Channel for sending frames from agent to viewers
pub type FrameSender = broadcast::Sender<Vec<u8>>;
pub type FrameReceiver = broadcast::Receiver<Vec<u8>>;
/// Channel for sending input events from viewer to agent
pub type InputSender = tokio::sync::mpsc::Sender<Vec<u8>>;
pub type InputReceiver = tokio::sync::mpsc::Receiver<Vec<u8>>;
/// Internal session data with channels
struct SessionData {
info: Session,
/// Channel for video frames (agent -> viewers)
frame_tx: FrameSender,
/// Channel for input events (viewer -> agent)
input_tx: InputSender,
input_rx: Option<InputReceiver>,
}
/// Manages all active sessions
#[derive(Clone)]
pub struct SessionManager {
sessions: Arc<RwLock<HashMap<SessionId, SessionData>>>,
agents: Arc<RwLock<HashMap<AgentId, SessionId>>>,
}
impl SessionManager {
pub fn new() -> Self {
Self {
sessions: Arc::new(RwLock::new(HashMap::new())),
agents: Arc::new(RwLock::new(HashMap::new())),
}
}
/// Register a new agent and create a session
pub async fn register_agent(&self, agent_id: AgentId, agent_name: String) -> (SessionId, FrameSender, InputReceiver) {
let session_id = Uuid::new_v4();
// Create channels
let (frame_tx, _) = broadcast::channel(16); // Buffer 16 frames
let (input_tx, input_rx) = tokio::sync::mpsc::channel(64); // Buffer 64 input events
let session = Session {
id: session_id,
agent_id: agent_id.clone(),
agent_name,
started_at: chrono::Utc::now(),
viewer_count: 0,
};
let session_data = SessionData {
info: session,
frame_tx: frame_tx.clone(),
input_tx,
input_rx: None, // Will be taken by the agent handler
};
let mut sessions = self.sessions.write().await;
sessions.insert(session_id, session_data);
let mut agents = self.agents.write().await;
agents.insert(agent_id, session_id);
(session_id, frame_tx, input_rx)
}
/// Get a session by agent ID
pub async fn get_session_by_agent(&self, agent_id: &str) -> Option<Session> {
let agents = self.agents.read().await;
let session_id = agents.get(agent_id)?;
let sessions = self.sessions.read().await;
sessions.get(session_id).map(|s| s.info.clone())
}
/// Get a session by session ID
pub async fn get_session(&self, session_id: SessionId) -> Option<Session> {
let sessions = self.sessions.read().await;
sessions.get(&session_id).map(|s| s.info.clone())
}
/// Join a session as a viewer
pub async fn join_session(&self, session_id: SessionId) -> Option<(FrameReceiver, InputSender)> {
let mut sessions = self.sessions.write().await;
let session_data = sessions.get_mut(&session_id)?;
session_data.info.viewer_count += 1;
let frame_rx = session_data.frame_tx.subscribe();
let input_tx = session_data.input_tx.clone();
Some((frame_rx, input_tx))
}
/// Leave a session as a viewer
pub async fn leave_session(&self, session_id: SessionId) {
let mut sessions = self.sessions.write().await;
if let Some(session_data) = sessions.get_mut(&session_id) {
session_data.info.viewer_count = session_data.info.viewer_count.saturating_sub(1);
}
}
/// Remove a session (when agent disconnects)
pub async fn remove_session(&self, session_id: SessionId) {
let mut sessions = self.sessions.write().await;
if let Some(session_data) = sessions.remove(&session_id) {
let mut agents = self.agents.write().await;
agents.remove(&session_data.info.agent_id);
}
}
/// List all active sessions
pub async fn list_sessions(&self) -> Vec<Session> {
let sessions = self.sessions.read().await;
sessions.values().map(|s| s.info.clone()).collect()
}
}
impl Default for SessionManager {
fn default() -> Self {
Self::new()
}
}