Overview
dots runs a large language model across a pool of Apple Silicon Macs that are already sitting on people's desks. Instead of renting a GPU or settling for a tiny model that fits on one laptop, dots splits a model layer by layer across several machines and runs each part in parallel. The result: you can serve models that are too large or too slow for any single device you own.
dots is a project I worked on with a friend. He led the ML engine; I owned the product: the web app, the live pool map, and the contribute flow that turns a stranger's idle Mac into a working node. The question that pulled me in was a simple one. Most of us are surrounded by capable, idle Apple Silicon, so what would it take to pool that compute into something that behaves like a single inference endpoint? I stayed close to the inference internals the whole way through, close enough to reason about the system end to end rather than just its surface.