How to Use a Local LLM for Writing on Your Mac

BLUF: To use a local LLM for writing on your Mac, install a model runner like Ollama, download a small writing-capable model, and pair it with an app that can act on the text you select. Everything then runs on-device — nothing you write or paste leaves your computer. This guide walks through what a local LLM actually is, what hardware you need, how to set one up, which model to pick, and how to use it across the apps you already type in every day.

What a local LLM for writing on your Mac is

A local LLM is a language model that runs entirely on your own machine instead of on a company's servers. You download the model file once, and after that it processes your text without any network request. That is the core difference from a cloud tool: there is no transmitted copy of your draft to log, leak, or train on. For anyone who writes about contracts, clients, unreleased products, or anything under an NDA, that on-device guarantee is the whole point.

What you need to run one

The practical bar is lower than most people expect. A Mac with Apple Silicon (M1 or newer) and at least 16GB of memory will run small writing models smoothly; 8GB works for the smallest models but feels tight. You also need disk space — a typical 7B–8B model is 4–6GB. Older Intel Macs can run small models too, just slower. If your machine clears that bar, a local LLM for writing on Mac is realistic for daily use, not just a demo.

How to set up a local LLM with Ollama

The simplest path is Ollama, an open-source runner that handles downloading and serving models. Install it, then in Terminal run a command like ollama pull llama3.1 to fetch a model. Ollama then exposes the model on your Mac so other apps can talk to it locally. That is the engine; you still need an interface that sends your selected text to it, which is the next step. For the bigger privacy picture, see why on-device AI on a Mac matters.

Choosing the right model for writing

Model choice is a trade between quality and speed. Smaller 7B–8B models (Llama 3.1 8B, Qwen, Mistral) respond quickly and handle rewriting, explaining, and short replies well. Larger models read more naturally on hard drafts but need more memory and feel slower. The honest advice: start small, judge it on your real work, and only step up if the output falls short. For most everyday writing, a small local model is closer to a cloud model than people assume.

Using a local LLM across your everyday apps

A model alone has no interface — it just waits for input. The piece that makes it useful is a tool that reads whatever text you select and routes it to your local model. Nugumi is a macOS menu-bar app that connects to a local Ollama model and works in any app with selectable text: Slack, Gmail, Notion, Safari, KakaoTalk, PDFs, code editors. Select a confusing passage and it explains the meaning in plain words; select your own rough draft and it polishes the tone; select an incoming message and it drafts a reply in your style. With a local model, all of it stays on your Mac.

Keeping cloud as an option

On-device should be your default for sensitive text, but you do not have to give up cloud quality entirely. The smartest setup lets you pick per document: the local model for anything confidential, a cloud model when you want maximum polish on a draft that carries no privacy stakes. Nugumi supports all three setups — a ChatGPT or Claude subscription, a fully local Ollama model, or your own API key — so you match the engine to the text instead of trusting one provider with everything. For a deeper look at the always-private route, read about a private AI writing assistant that works offline.

A short checklist before you commit

Before you settle on a setup, confirm a few things. Your Mac meets the memory bar so responses feel fast, not painful. The model handles your real writing tasks, not just toy prompts. The tool you pair it with works in every app you type in, not one editor window. And you can switch between local and cloud per document when the stakes change. If all four hold, you have a local LLM for writing on your Mac that protects your work without slowing you down. You can download Nugumi and test the local path on real work in a few minutes.