Photoshoot

Why it was hard

Themes change the app's entire chrome and filters change what the GPU does to every pixel. Both are things a user should be able to download from a stranger and try, and both are the classic route to arbitrary execution. A theme that can set a CSS value can smuggle url(javascript:…), @import or a script tag; a filter that can supply shader text can rewrite the pipeline. The desktop app is an Electron process, so a mistake here is not a defaced page — it is a foothold.

Options considered

• Ship only built-in themes and filters, and drop the community format entirely
• Accept CSS text from a manifest and sanitize it on the way in
• Accept GLSL snippets and validate the source before compiling it
• Reduce both formats to whitelisted, typed values with no free text anywhere

What I built

A theme manifest may set only the 59 whitelisted CSS custom properties — 39 colour, 9 length, 5 shadow, 2 font and 4 texture. Each is matched against a per-kind regular expression, then screened against 24 forbidden substrings including javascript:, expression(, @import, <script, url(, http://, file:, blob:, backslash escapes, braces and semicolons. Texture tokens can never be set from a manifest string at all; they come only from bytes the importer has already verified by magic number. A filter is 11 numbers, each hard-clamped to a fixed minimum, maximum and default, plus an optional lookup table. Every value is validated twice — once by the importer, and again by the renderer immediately before it reaches the CSSOM or a GL uniform.

What it cost

Community themes can only do what the whitelist anticipated. Anything genuinely new needs a new token, a new regular expression and a new release, so the format cannot be extended by the people using it. That is the price of it being data.

How I know it works

44 adversarial assertions run over the filter validator: out-of-range values, NaN, Infinity, wrong types, unknown keys, injected strings, junk manifests, and the renderer's re-clamp of a store that was edited on disk. The invariant is written into the shader file so it survives the next reader: no part of a manifest ever becomes shader source, JavaScript, CSS or HTML.

Why it was hard

Imported filters need to look different from one another — brightness, contrast, temperature, hue rotation, film grain, a colour lookup table. The obvious implementation generates shader source per filter, which puts untrusted data on the path to the GPU compiler and adds a compile stall the first time each one is selected.

Options considered

• Generate GLSL per filter from the manifest and compile on first use
• Precompile one program per known filter shape and switch between them
• Compile one fixed program and drive every filter entirely from uniforms

What I built

All imported filters resolve to a single program: the effect id is mapped with `const key = id.startsWith('custom:') ? 'customfilter' : id`, so every custom filter shares one compiled shader. The difference between two filters is three vec4 uniforms and an optional LUT texture. The shader runs brightness, contrast, saturation, temperature, tint, gamma, fade, a hue rotation about the (1,1,1) axis, a 64-level lookup table with blue-channel interpolation, vignette and grain — in that fixed order.

What it cost

Every imported filter pays for every stage whether it uses it or not, and a filter can only ever be a point inside that fixed parameter space. Generated shaders would be cheaper per filter and unbounded in what they could express; this trades expressiveness for a surface area of eleven range checks.

How I know it works

gamma is clamped to a floor of 0.2 so the shader's 1/gamma can never divide by zero, and that floor is asserted directly in the suite. A lookup table must be a real 512×512 PNG, checked by magic bytes and by the width and height read out of the IHDR chunk at byte offsets 16 and 20 — never by file extension. Non-square images, truncated files, oversized files and a JPEG wearing a .png name are all covered.

Why it was hard

The camera <video> element is deliberately mounted one pixel wide at opacity 0, because pixels flow through WebGL rather than through the tag. Driving the render loop from requestVideoFrameCallback is the textbook choice — it fires once per decoded video frame, exactly the cadence a texture upload wants. On some browser, GPU and operating-system combinations it never fires at all for an element in that state. getUserMedia had resolved, the tracks were live, and the app looked broken.

Options considered

• Mount the video visibly and hide it behind the canvas so rVFC has a painted element
• Keep rVFC and add a watchdog that restarts the loop when no frame arrives
• Drive the loop on requestAnimationFrame and stop depending on rVFC entirely

What I built

The loop always ticks on requestAnimationFrame, which fires at display refresh whether or not a video frame was presented, and the texture upload takes whatever the video element currently holds. The reasoning is recorded as an eight-line comment at the scheduling call itself, so the next reader does not reintroduce the optimisation.

What it cost

On a camera running below display refresh, the same frame is uploaded more than once and that GPU work is wasted. Predictable liveness was worth more than avoiding redundant uploads.

How I know it works

The debug overlay reports the active backend, which now reads `WebGL2 · rAF`. The fix is a discrete commit rather than a change bundled into unrelated work.

Why it was hard

Background replacement means deciding, per pixel and per frame, what is the person and what is the room. The modern answer is a segmentation model, which would have meant a second on-device model to download and a second inference budget on top of the face mesh. The naive alternative — reading pixels back from the GPU at full resolution and differencing them on the CPU — stalls the pipeline every frame.

Options considered

• Add a second MediaPipe model for selfie segmentation
• Difference every frame against a reference at full resolution on the CPU
• Difference against a low-resolution reference and let the compositor do the upscaling

What I built

The user captures a reference frame of the empty scene at 192 px wide. Each frame is differenced against it with a sum-of-channels threshold derived from a user tolerance setting — the only CPU pixel loop in the application, deliberately confined to that resolution. The resulting low-resolution alpha mask is upscaled by the GPU during destination-in compositing, which softens the matte edge for free rather than requiring a separate feather pass. The composited canvas is then fed back into the GL pipeline as the source, so effects and mirroring still apply on top.

What it cost

It asks the user to step out of frame once, and it degrades if the camera or the lighting moves. A segmentation model would need neither. It also means the six backdrops had to be drawable rather than photographable — they are generated procedurally with Canvas2D and cached per size, so no image assets ship.

How I know it works

The cost is bounded by construction rather than by measurement: the loop is fixed at a 192 px sample regardless of camera resolution, and the upscale is the compositor's work, not the CPU's. There is no benchmark in the repository, so I have no frame-rate figure to publish for it and will not quote one.

Why it was hard

The desktop app has Node, IPC, a filesystem and a main process. The browser has none of those and cannot be given them. Keeping two builds in step usually degrades one of two ways: a shared core with two thin apps around it that quietly drift apart, or platform checks sprinkled through the renderer that multiply with every feature.

Options considered

• Two codebases kept in step by hand
• One codebase with `if (isElectron)` branches inside the renderer
• A bridge interface the renderer talks to, implemented once per runtime

What I built

The renderer only ever calls PhotoshootBridge, a twenty-method interface declared in src/shared/ipc-contract.ts alongside the twenty IPC channel names. preload.ts implements it over contextBridge and ipcRenderer; src/web/shim.ts implements the same twenty methods over IndexedDB, localStorage and a download anchor. src/shared/ imports neither Node nor the DOM, so it bundles unchanged into the main process, the preload, the renderer and the web shim. The web build bundles src/renderer/main.ts verbatim and only swaps the script tags and sets data-platform="web".

What it cost

The interface is the lowest common denominator of the two runtimes, so capabilities that only make sense on the desktop — revealing a file in its folder, for example — still have to exist in the browser and do the nearest native equivalent. Adding one feature means touching the contract, both implementations and the preload bridge: four files for one capability.

How I know it works

`tsc --noEmit` runs clean with strict mode on, zero diagnostics, which is what actually enforces that both implementations satisfy the interface. The deployed web app references exactly the hashed filenames produced by the local build, so the two runtimes are demonstrably built from the same renderer source.