feat(agent,server): v2 secure-session-core Task 7 - HW H.264 + negotiated raw fallback
All checks were successful
Build and Test / Build Agent (Windows) (push) Successful in 6m57s
Build and Test / Build Server (Linux) (push) Successful in 10m23s
Build and Test / Security Audit (push) Successful in 4m15s
Build and Test / Build Summary (push) Successful in 9s

SPEC-002 Phase 1 Task 7 (the last), code-reviewed APPROVED, locally verified
(cargo fmt + clippy -D warnings exit 0 + cargo test --workspace 89 pass + build).

- Encoder trait + factory: RawEncoder (salvaged, UNCHANGED) and H264Encoder,
  selected by negotiation; factory falls back to raw on H.264 init failure.
- Negotiation: agent advertises supports_h264 (MFTEnumEx HW probe, cached) in
  AgentStatus; server picks the codec via select_video_codec(supports, prefer)
  and stamps StartStream.video_codec; agent re-guards on local HW. Policy
  constant DEFAULT_PREFER_H264 = false, so RAW is negotiated for every session
  today - H.264 stays dormant until live hardware validation (Task 8).
- MF H.264 encoder (h264.rs, FIRST-CUT / compile-verified-only): HW encoder MFT,
  BGRA->NV12 (color.rs, unit-tested), sync drain, fall-back-to-raw on any failure.
- Viewer H.264 decoder (decoder.rs, FIRST-CUT): MF decoder on a dedicated COM
  thread; drops+logs on failure, raw render path untouched.
- proto additive: VideoCodec enum, StartStream.video_codec=3,
  SessionResponse.video_codec=5, AgentStatus.supports_h264=11.
- Raw+Zstd path byte-for-byte unchanged; remains the guaranteed default/fallback.

Review confirmed unsafe impl Send for H264Encoder is sound (single-owned &mut on
the block_on thread; session future never spawned) and every MF failure degrades
to raw. H.264 is NOT claimed functional - compile/clippy/build-verified only;
live validation + force-IDR + the no-spawn-invariant doc are Task 8 go-live gates.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-30 10:35:04 -07:00
parent bb73ba667f
commit f9bdecbfdb
12 changed files with 1885 additions and 23 deletions

View File

@@ -3,6 +3,8 @@
//! This module provides the viewer functionality for connecting to remote
//! GuruConnect sessions with low-level keyboard hooks for Win key capture.
#[cfg(windows)]
mod decoder;
mod input;
mod render;
mod transport;
@@ -31,6 +33,72 @@ pub enum InputEvent {
SpecialKey(proto::SpecialKeyEvent),
}
/// Spawn the dedicated H.264 decode worker thread (Task 7, Windows only).
///
/// Returns a sender for `(h264_access_unit, pts_100ns)`. The worker lazily
/// creates the Media Foundation decoder on the first frame; if creation fails it
/// logs once and then silently drops subsequent frames (the raw render path is
/// never affected). Each decoded frame is converted to BGRA and delivered to the
/// viewer as an uncompressed `FrameData`, reusing the existing render path.
#[cfg(windows)]
fn spawn_h264_decode_worker(
viewer_tx: mpsc::Sender<ViewerEvent>,
) -> std::sync::mpsc::Sender<(Vec<u8>, i64)> {
let (tx, rx) = std::sync::mpsc::channel::<(Vec<u8>, i64)>();
std::thread::Builder::new()
.name("gc-h264-decode".to_string())
.spawn(move || {
let mut decoder: Option<decoder::H264Decoder> = None;
let mut init_failed = false;
while let Ok((data, pts)) = rx.recv() {
if init_failed {
continue;
}
if decoder.is_none() {
match decoder::H264Decoder::new() {
Ok(d) => {
info!("H.264 decoder initialized (Media Foundation)");
decoder = Some(d);
}
Err(e) => {
error!(
"H.264 decoder init failed: {e:#}; H.264 frames will be dropped"
);
init_failed = true;
continue;
}
}
}
let dec = decoder.as_mut().expect("decoder present after init");
match dec.decode(&data, pts) {
Ok(Some(decoded)) => {
let frame = render::FrameData {
width: decoded.width,
height: decoded.height,
data: decoded.bgra,
compressed: false, // already BGRA
is_keyframe: false,
};
if viewer_tx.blocking_send(ViewerEvent::Frame(frame)).is_err() {
// Viewer closed; stop the worker.
break;
}
}
Ok(None) => { /* decoder buffering; no output this tick */ }
Err(e) => {
warn!("H.264 decode error: {e:#}");
}
}
}
})
.expect("failed to spawn H.264 decode worker thread");
tx
}
/// Run the viewer to connect to a remote session
pub async fn run(server_url: &str, session_id: &str, api_key: &str) -> Result<()> {
info!("GuruConnect Viewer starting");
@@ -77,13 +145,23 @@ pub async fn run(server_url: &str, session_id: &str, api_key: &str) -> Result<()
}
});
// H.264 decode worker (Task 7, Windows only). The Media Foundation decoder
// wraps COM interfaces with thread affinity, so it runs on a DEDICATED OS
// thread (not a tokio task, which can migrate across workers at await
// points). The receive task forwards H.264 access units to it over a std
// channel; the worker decodes to BGRA and pushes a FrameData back through
// the viewer channel via `blocking_send`. On decoder-init failure the worker
// logs and drops H.264 frames (the raw path is unaffected).
#[cfg(windows)]
let h264_tx = spawn_h264_decode_worker(viewer_tx.clone());
// Spawn task to receive messages from server
let viewer_tx_recv = viewer_tx.clone();
let receive_task = tokio::spawn(async move {
while let Some(msg) = ws_receiver.recv().await {
match msg.payload {
Some(proto::message::Payload::VideoFrame(frame)) => {
if let Some(proto::video_frame::Encoding::Raw(raw)) = frame.encoding {
Some(proto::message::Payload::VideoFrame(frame)) => match frame.encoding {
Some(proto::video_frame::Encoding::Raw(raw)) => {
let frame_data = render::FrameData {
width: raw.width as u32,
height: raw.height as u32,
@@ -93,7 +171,23 @@ pub async fn run(server_url: &str, session_id: &str, api_key: &str) -> Result<()
};
let _ = viewer_tx_recv.send(ViewerEvent::Frame(frame_data)).await;
}
}
Some(proto::video_frame::Encoding::H264(enc)) => {
// Forward to the decode worker (Windows). On other
// platforms H.264 is never negotiated, so this is dead.
#[cfg(windows)]
{
if h264_tx.send((enc.data, enc.pts)).is_err() {
warn!("H.264 decode worker unavailable; dropping frame");
}
}
#[cfg(not(windows))]
{
let _ = enc;
}
}
// VP9/H265 not implemented on the viewer (raw + H.264 only).
_ => {}
},
Some(proto::message::Payload::CursorPosition(pos)) => {
let _ = viewer_tx_recv
.send(ViewerEvent::CursorPosition(pos.x, pos.y, pos.visible))