Rendering Pipeline

From N64 framebuffer to your screen — with shaders, upscaling, and zero main-thread blocking.

---

Current State (What Exists Today)

All browser N64 emulators use HLE (High-Level Emulation) for graphics:

HLE recognizes known Nintendo microcodes (Fast3D, F3DEX, F3DEX2) and translates them directly to OpenGL draw calls. It's fast but:

• Fails on games with custom microcodes (Rogue Squadron, Factor 5 titles)
• Not pixel-accurate to real hardware
• Can have texture filtering differences

For THPS and 90%+ of commercial games, HLE works great.

---

Our Rendering Architecture

---

OffscreenCanvas: Why It Matters

Traditional approach (every current browser emulator):

// Main thread — competes with everything else
const canvas = document.getElementById('game');
const gl = canvas.getContext('webgl2');
// ... emulation step ...
gl.texImage2D(..., framebuffer);  // BLOCKS main thread
gl.drawArrays(...);               // BLOCKS main thread

Our approach:

// Main thread — just transfers the canvas once
const canvas = document.getElementById('game');
const offscreen = canvas.transferControlToOffscreen();
gfxWorker.postMessage({ canvas: offscreen }, [offscreen]);

// GFX Worker — dedicated thread, never blocks anything
self.onmessage = (e) => {
  const gl = e.data.canvas.getContext('webgl2', {
    desynchronized: true,    // Don't sync with compositor
    powerPreference: 'high-performance'
  });
  // Render loop runs here, completely independent
};

Measured improvement: 0.20ms per frame vs 0.80ms on main thread (4x faster frame delivery).

---

Post-Processing Shaders

Since we own the rendering pipeline, we can add GPU-accelerated post-processing:

CRT Shader (Retro Mode)

// Scanline + curvature + bloom
vec2 uv = curved_uv(texCoord, 0.02);
vec3 color = texture(framebuffer, uv).rgb;
color *= scanline(uv.y, 240.0);
color += bloom(framebuffer, uv) * 0.15;
color = apply_vignette(color, uv);

Clean Upscale (HD Mode)

// Bilinear with edge-aware sharpening
vec3 color = texture(framebuffer, texCoord).rgb;
vec3 sharp = sharpen_edge_aware(framebuffer, texCoord, 0.5);
color = mix(color, sharp, sharpness);

Available Filters

Filter	Description	GPU Cost
None	Raw pixels, nearest-neighbor	Minimal
Bilinear	Smooth scaling	Low
CRT	Scanlines + curvature + bloom	Medium
CRT-Royale	Accurate CRT phosphor simulation	High
xBR	Edge-smoothing upscale	Medium
Sharp	Edge-aware sharpening	Low

---

Resolution Scaling

The N64 renders at 320x240 (or 240x160 for some games). We can upscale:

Scale	Output	Target Display
1x	320x240	Retro purist
2x	640x480	Standard
3x	960x720	HD
4x	1280x960	Desktop fullscreen

Scaling is done in the shader pipeline — zero cost to emulation performance.

---

Future: WebGPU + ParaLLEl-RDP

The holy grail of N64 rendering in a browser:

ParaLLEl-RDP achieves bitexact accuracy with real N64 hardware using Vulkan compute shaders. If ported to WebGPU (WGSL shaders), we'd have:

• Pixel-perfect rendering for every game
• HD resolution upscaling (2x-8x) with proper filtering
• No microcode compatibility issues (LLE executes real RSP programs)
• HD texture pack support

WebGPU Browser Support (2026)

Browser	Status
Chrome 113+	Shipped (April 2023)
Edge 113+	Shipped
Safari 18+	Partial
Firefox	In progress

The Port Challenge

ParaLLEl-RDP's Vulkan shaders need translation to WGSL. Key considerations:

• ~30 compute shaders handling different RDP modes
• Subgroup operations (partially available in WebGPU)
• Storage buffer access patterns
• Synchronization between dispatch calls

This is a significant engineering effort but would make N64.wasm the only browser emulator with hardware-accurate rendering. Nobody else is attempting this.

---

Clip Recording Integration

The rendering pipeline integrates with MediaRecorder for gameplay capture:

// Capture the canvas stream
const stream = canvas.captureStream(30); // 30fps recording
const recorder = new MediaRecorder(stream, {
  mimeType: 'video/webm;codecs=vp9',
  videoBitsPerSecond: 5_000_000
});

// Rolling 30-second buffer (DVR mode)
const chunks = [];
recorder.ondataavailable = (e) => {
  chunks.push(e.data);
  if (chunks.length > 30) chunks.shift(); // Keep last 30 seconds
};
recorder.start(1000); // Chunk every second

User hits "Clip That" → last 30 seconds become a shareable WebM.