Personal project / In development
Native H.264 engine, PipeWire capture and NVIDIA/Intel acceleration with rendering outside the WebView.
Screen sharing separates the management interface from video processing. The Linux/Wayland path combines system portal capture, H.264 encoding, WebRTC transport and presentation in a native window.
Development of the C++26 engine, capture, encoders, decoders, media transport and SDL3 client. Next.js/Tauri launches the C++ executable as a sidecar through a Rust command; Python/asyncio/websockets handles SDP and ICE rendezvous. The engine structure makes frame movement, memory transfers and synchronization points explicit.
NVIDIA MemFd alternative: mmap → CPU libswscale (RGB/BGRA → NV12) → av_hwframe_transfer_data to the GPU. Mapping support depends on formats and drivers.
NVENC configures a 5 Mb/s bitrate, zero B-frames and a 2 s GOP. RTP timestamps advance at 90 kHz according to FPS.
std::mutex and av_frame_clone protect frame exchange. CUDA completion precedes SDL drawing; RAM paths use SDL_UpdateNVTexture. NV12 planes are copied between surfaces in GPU memory.
Signaling exchanges metadata; video uses the media channel between peers.
Qt6/DBus accesses the XDG ScreenCast portal; PipeWire delivers buffers on its own thread. The engine passes FD, dimensions, stride, SPA format and DRM modifier to the encoder, then returns the buffer after the callback. AVDRMFrameDescriptor describes the DMA-BUF. Duplicating the FD retains the buffer reference during processing.
CudaEncoder uses FFmpeg h264_nvenc, a CUDA context and NV12 surfaces. IntelIrisEncoder configures h264_qsv and a pool of 32 QSV surfaces. Both contain DRM PRIME mapping through av_hwframe_map. In NVIDIA’s MemFd fallback, libswscale converts pixels on the CPU before av_hwframe_transfer_data uploads them; the next step is to move all color conversion to the GPU.
CudaDecoder selects h264_cuvid and negotiates AV_PIX_FMT_CUDA for GPU-resident frames. The Intel module uses h264_qsv and includes DRM PRIME mapping, but also transfers the frame to RAM. SoftwareDecoder uses AV_CODEC_ID_H264 and libswscale for NV12. Automatic codec selection and integration across variants are next steps.
std::mutex serializes frame replacement and reading; av_frame_clone retains references. The renderer registers SDL3’s OpenGL textures, maps Y/UV and uses cudaMemcpy2DToArrayAsync with cudaMemcpyDeviceToDevice. After unmapping, cudaStreamSynchronize(0) waits for completion before drawing. This wait coordinates the CPU thread with completion of GPU copies before presentation.
The source uses libdatachannel, unlike the libwebrtc named in the initial blueprint. H.264 NAL units are packetized into RTP, with host RTCP Sender Reports and NACK; the client depacketizes and requests a keyframe when the track opens. Timestamps advance by 90,000/FPS. A condition_variable waits for the track to open before sending: network coordination, separate from local frame availability in the renderer.
FFmpeg: libavcodec/libavutil for codecs and hardware contexts, libswscale for CPU conversion; libavfilter is included in the filter layer dependencies. PipeWire/SPA, Qt6/DBus, libdrm and EGL/GBM cover Linux capture and formats. CUDA and QSV/libmfx/libva support hardware variants; SDL3/OpenGL presents frames, libdatachannel transports media and nlohmann/json serializes messages.
In development. Next steps: consolidate codec selection, move color conversion to the GPU, expand macOS support and add audio and remote input.
Let’s talk about opportunities, projects and the engineering challenges ahead.
Connect on LinkedIn