WiCi, the startup behind the WiCi One wireless GPU, has launched WiCi Infer, a free desktop app that it says can run 800B-class AI models on everyday computers, even when the model is bigger than the machine’s memory.
The trick is in how the model is loaded. Today’s largest models only use a small fraction of their total size to generate each word of output. Rather than loading the whole thing into memory, WiCi Infer pulls in just the parts it needs, as it needs them, from the computer’s SSD into RAM or GPU memory.
In WiCi’s own testing, that allowed Meta’s 816-billion-parameter Muse Spark Open model to run on a MacBook Pro with a 1TB SSD and just 24GB of RAM, with 48GB recommended. Without WiCi Infer, WiCi says that model would need 256GB or more.
WiCi also claims big speed gains. On an RTX 5090 desktop with 32GB of RAM and 32GB of VRAM, it says WiCi Infer ran the DeepSeek-V4.1-Flash model at 7.27 tokens per second, 8.4 times faster than the popular llama.cpp on the same machine, and around three times faster on a 48GB M5 Max MacBook Pro.
Why photographers should care
For photographers, the attraction of local AI is privacy. Client shoots, unreleased work, model releases and personal archives are exactly the kind of files many of us don’t want to upload to a cloud service just to get help from AI.
WiCi Infer is designed to keep everything on your machine. You can connect the writing, coding and document tools you already use through a custom model endpoint, so prompts and files never leave your computer. That opens up private help with the admin side of photography: drafting captions and image descriptions, writing alt text, keywording notes, client emails, shoot briefs and blog posts, all without your work leaving the studio.
The bigger question for us is how well this approach will handle image-aware tasks such as tagging, culling and describing photos, which depend on vision models. WiCi hasn’t detailed that yet, and it’s something we’d like to test. It also makes a fast SSD more important than ever, since the drive becomes part of the AI pipeline.
It’s a neat fit with the AI mini PCs we’ve been covering. Many photographers have bought machines with 32GB or 64GB of memory for editing, and WiCi Infer suggests those could run models that would normally need workstation-class hardware.
The app has a graphical interface, so there’s no command line, and the inference engine behind it is open source from day one. We previously covered the WiCi One wireless GPU.
WiCi Infer key features
- Runs 800B-class open-weight models by streaming model parts from SSD
- Full-quality models without aggressive compression
- Claimed up to 8.4x faster than llama.cpp on an RTX 5090 desktop
- Private by default: prompts and files stay on your machine
- Custom model endpoint for existing writing, coding and document tools
- Graphical interface; open-source inference engine
WiCi Infer price and availability
WiCi Infer is a free download, available now from the WiCi website.




0 Comments