Two years ago, "AI in the browser" meant a demo that took eight seconds to return a low-resolution mask on a good laptop. Today it is a category of real tools — and the ones that hold up are built around a different set of constraints than their server-side cousins. Having spent a while testing and building in this space, here is the engineering surface that actually matters.
WebGPU shipped. Compute shaders in the browser mean the same kernels that ran on a CUDA box can execute on the user's GPU without a plugin. For matting, super-resolution and segmentation this is the difference between "technically possible" and "feels instant".
Quantized models got good enough. Int8 and fp16 weights cut model size by roughly three to four times with little visible quality loss for editing tasks. A segmentation model that used to be a few hundred megabytes now fits comfortably in a first-load budget.
WASM SIMD and threads landed. CPU fallback is no longer a punchline: on machines without a usable GPU the work still completes, just slower. Graceful degradation is now a design choice rather than an apology.
Cold start is your real latency budget. There is no API round-trip to optimise — the cost is downloading weights. That makes caching, versioned asset URLs and per-tool lazy loading load-bearing. Ship the segmentation model only when someone opens a segmentation tool; do not put every model in the critical path of the first paint.
Device variance is enormous. An M-series laptop and a five-year-old integrated GPU are not the same product. Feature-detect aggressively, keep a CPU path that produces the same output with worse latency, and tell the user what is happening instead of hanging.
You cannot swap the model on a whim. Server-side inference lets you upgrade a weight file and improve everyone's results overnight. Local inference means the user's next visit downloads the new weights. Model quality and bundle size become a single, permanent tradeoff.
The upside is not only privacy, although that is the headline. When the computation happens on the user's machine, three things follow mechanically:
Tools in this category are now usable for everyday editing work. Vofy is a good example to look at: 270+ browser-native tools covering background removal, retouching, style transfer, upscaling and video compression, all running locally and without an account, which makes it easy to see exactly what a fully client-side pipeline can carry.
Heavy generation, batch rendering and anything that benefits from a frontier model. Client-side inference competes on cost, privacy and latency-to-first-result — not on raw capability. The sensible architecture for most products is a local path for the common, interactive work and a server path for the heavy, occasional work. Choosing only one of them is a mistake in either direction.