Swiftlet Runs an 80B LLM in 4.3 GB of RAM on a Mac. Here Is How to Build on It.

By Umair Ali Sadaqat · Updated · Editorial policy

A new open-source project called Swiftlet uses aggressive quantization to run Qwen3-80B in 4.3 GB of RAM on a MacBook, and Qwen3-35B on an iPhone. This guide covers what that actually means for developers, where the quality tradeoffs are, and how to wire a local LLM endpoint into a real product today.

Swiftlet hit the Hacker News front page on August 3, 2026 with 260 points and 116 comments. The pitch: run Qwen3-80B, a frontier-class large language model with 80 billion parameters, in 4.3 GB of RAM on a MacBook. A 35B model runs on an iPhone.

TL;DR: Swiftlet (github.com/leonickson1/Swiftlet) uses extreme quantization to fit models that normally need 160 GB of memory into consumer Apple hardware. The quality tradeoff is real but narrower than you would expect. If you want to build a product powered by a local LLM with zero API costs, this is the most accessible way to start. Wire a We.Inc app to the local endpoint, publish it, and your users never know the model runs on your laptop.

What Swiftlet Actually Does

The core trick is not magic. It is quantization taken further than most tools go.

The math: Qwen3-80B at FP16 (the standard precision for inference) needs roughly 160 GB of memory. At the commonly-used 4-bit quantization (GGUF Q4_K_M), it needs about 40 GB. Swiftlet pushes to roughly 0.4 bits per parameter on average, using grouped quantization with importance-weighted bit allocation. The most critical weights (attention heads, first and last layers) get more bits; the bulk of feed-forward layers get fewer.

The hardware target: Apple Silicon. Swiftlet is written in Swift and uses Metal for GPU acceleration. M1/M2/M3/M4 Macs with unified memory are the sweet spot because the GPU and CPU share the same RAM pool, which avoids the memory-copy bottleneck that limits NVIDIA-based local inference.

What you get: a local HTTP server (typically localhost:8080) that accepts prompts and returns completions. Token generation speed on an M3 MacBook Pro is reported at 8-12 tokens per second for the 80B model, which is usable for interactive applications but not instant.

Where the Quality Tradeoff Lands

Extreme quantization is not free. Here is where it holds up and where it does not, based on the HN discussion and early testing:

Holds up well:

  • Conversational Q&A and chat interfaces
  • Content summarization and rewriting
  • Classification, tagging, and extraction
  • Instruction following for structured outputs (JSON, forms)

Degrades noticeably:

  • Complex multi-step code generation (the 35B at 4-bit often beats the 80B at sub-2-bit here)
  • Mathematical reasoning with exact answers
  • Long structured documents where coherence drifts
  • Tasks requiring precise world knowledge (dates, statistics)

The practical takeaway: if your product is a chatbot, a summarizer, or a content tool, the 80B at extreme quantization is surprisingly capable. If your product generates code or does math, test the 35B at higher quantization instead. More bits per parameter on a smaller model often beats fewer bits on a bigger one.

How to Wire a Local Swiftlet Endpoint Into an App

The build path is straightforward. You do not need a cloud API key or a billing account.

Step 1: Install and start Swiftlet. Clone the repo, download the quantized Qwen3-80B weights (the 4.3 GB file), and run the server. On a Mac with 8 GB of RAM or more, it starts in about 30 seconds.

Step 2: Confirm the endpoint works. Send a test prompt to localhost:8080 with curl or any HTTP client. You should get a completion back in a few seconds.

Step 3: Build the app layer. This is where you spend most of your time. You need a frontend that sends user input to the local endpoint and displays the response. With We.Inc, describe the app you want ("a writing assistant that takes a paragraph and returns three rewritten versions") and the AI builds a working React app with a live preview. Then edit the API call to point at your Swiftlet endpoint.

Step 4: Publish or export. You can publish directly from We.Inc to a custom domain (the app calls your local endpoint at runtime), or export the code to GitHub and deploy it yourself. The code is standard React, Vite, and TypeScript with no proprietary dependencies.

The result: a production-quality web app powered by a local 80B model, with zero ongoing API costs.

When Local Inference Makes Sense (and When It Does Not)

Local wins when:

  • You want zero marginal cost per query (no API billing)
  • Your data cannot leave the device (privacy-sensitive applications: medical, legal, financial)
  • You are building a demo or prototype and do not want to set up cloud API credentials
  • You are in a location with unreliable internet

Cloud wins when:

  • You need to serve multiple concurrent users (a laptop runs one inference stream at a time)
  • You need the highest possible quality (full-precision frontier models still beat extreme quantization)
  • Your users are on devices that cannot run Swiftlet (Windows, Linux, mobile web)
  • You need uptime guarantees

The honest middle ground: use Swiftlet for development and demos, then switch to a cloud endpoint (OpenRouter, Together AI, the Qwen API) for production scale. The app code is the same either way; you just change the endpoint URL.

What This Means for the Category

Running an 80B model on a laptop is a milestone, not because the quality matches a datacenter, but because it removes the API-key barrier entirely. Anyone with a MacBook can now run a frontier-class model locally and build on top of it.

For developers who have been building AI-powered tools, the cost equation changes. API bills for prototype-stage products have been a real friction point. Swiftlet eliminates that friction during development and testing. The model runs on your machine, it costs nothing per query, and the inference is fast enough for interactive use.

Combined with a builder like We.Inc that handles the frontend and hosting, the path from "I have an idea for an AI tool" to "it is live on a custom domain" is now: install Swiftlet, describe the app, point it at localhost, publish. That is a meaningful reduction in the number of steps between idea and product.

Start building with We.Inc for free and wire it to your local Swiftlet endpoint. Export the code to GitHub whenever you want.

Frequently asked questions

What is Swiftlet?

Swiftlet is an open-source project (github.com/leonickson1/Swiftlet) that runs large language models on Apple hardware using extreme quantization techniques. It can run Qwen3-80B in 4.3 GB of RAM on a Mac and Qwen3-35B on an iPhone. It is written in Swift and optimized for Apple's Metal GPU framework.

How does Swiftlet fit an 80B model into 4.3 GB of RAM?

Swiftlet uses aggressive quantization, likely sub-2-bit, to compress the model weights far below their original size. An 80B parameter model at full precision (FP16) requires roughly 160 GB of memory. At 4-bit quantization it needs about 40 GB. At the 4.3 GB level Swiftlet achieves, the quantization is extreme, around 0.4 bits per parameter on average, which uses techniques like grouped quantization and importance-weighted bit allocation to preserve quality on the most critical weights.

Is the quality good enough for production use?

It depends on the task. Extreme quantization preserves the model's general knowledge and instruction-following ability well, but degrades performance on tasks requiring precise numerical reasoning, complex multi-step logic, or long structured outputs. For conversational interfaces, content generation, and classification, quality is surprisingly usable. For code generation or complex reasoning, a less-quantized smaller model (like Qwen3-35B at 4-bit) may produce better results. Benchmark before shipping.

Can I build an app that uses a local LLM running via Swiftlet?

Yes. Swiftlet exposes a local HTTP endpoint (typically on localhost:8080) that accepts prompts and returns completions. Any web app can call that endpoint. With We.Inc, you can describe your app in plain English, get a working React app immediately, add a fetch call to the local Swiftlet endpoint, and have a functioning product in minutes. The generated code is yours to export to GitHub, so you are not locked into any platform.

More in Blog

Start building

More on this topic

How we research, test and update this page: our editorial policy. We.Inc is our own product.

Product

Who It's For

Features

Resources

Company

View Sitemap