KoboldCpp review: check where your story actually runs

KoboldCpp suits writers who want a local model and direct control over a continuing story. It is free software; you supply the model and computing resources. Check the selected backend before treating a session as private: KoboldAI Lite can also send requests to external services. We checked its hosted interface, not local inference.

The same Lite interface can lead to three different places

KoboldCpp runs the model server; KoboldAI Lite is its bundled browser interface. The official setup guide separates the executable, model selection and browser UI. Opening a chat window alone does not show that a model is running on your computer.

Your own computer

A compatible model is loaded by your local KoboldCpp process. Check the server's launch address and the frontend's selected provider.

A remote server or API

The browser can use another machine or an external provider. Its operator, billing and processing terms matter even if the interface looks familiar.

AI Horde

Volunteer workers supply the computing. This is a different route from generating on your own hardware.

The README’s external-provider section also documents running without a local model. Even a locally served frontend is not proof of local inference. Confirm both the loaded model and the selected connection.

For a private fictional writing project, I would identify the route before typing the story. A familiar model name in a hosted worker list says nothing about who operates the hardware. Our check did not inspect actual network traffic or establish a privacy guarantee.

Memory and Author’s Note solve different writing problems

In Classic Lite, open Context → Memory. We entered a short canon note about a fictional 35-year-old archivist and a brass key, then put a scene-direction sentence in Author’s Note. After OK, closing and reopening Context retained both fields.

That verified editable setup, not recall. We sent no generation request and did not test retention after reload or a new session.

The official memory guide explains the distinction:

• Memory: recurring setting and character facts, inserted near the prompt’s start.
• Author’s Note: current scene or style direction, inserted nearer the end; strength changes placement.
• World Info: conditional reference material activated by matching keys.

My suggestion for a long story is to separate facts from direction. Keep “the brass key opens the west archive” in a compact canon note; keep “stay in the archive and use concise dialogue” in the current scene note. When a response drifts, change the relevant control rather than rewriting both at once.

Memory occupies context space. Pasting an entire story bible there leaves less room for the recent scene; the guide recommends keeping it short. These controls supply text to the model, not an independently verified long-term memory system.

Size the first writing session around context, not just the model file

A model file’s size is only a starting point. The current README separates model and context requirements; the official help page distinguishes load errors from backend-selection problems. I would troubleshoot in this order:

1. Keep the model filename, context limit and launch configuration in the test record.
2. If loading fails, try a smaller model or shorter context before changing many acceleration settings.
3. If generation is unexpectedly slow, check the selected hardware backend and offloaded layers.
4. If the reply is incoherent, check the chat template before blaming writing quality.

Disk size is not a RAM or VRAM requirement. We did not load a model or measure speed, so this review does not promise that a particular file will fit your machine. The model’s license and behavior also need their own review.

A newbie template can fetch more than a text model

The model-selection guide says templates download their files when you launch; its Chatbot templates also add voice and vision features. A portable executable is not the complete model bundle.

For a text-only writing trial, inspect the template’s scope or choose a compatible text GGUF first. Add speech and images only when you need them. This helps distinguish expected model downloads from a stalled first launch. We did not execute a template.

The current release matters if macOS stopped using the GPU

On October 25, GitHub marked 1.123.1, published October 13, as the latest release. Its hotfix note addresses a regression where macOS builds were not using the GPU. If that matches your symptom on an older build, check the version before treating poor speed as a hardware limit. We did not reproduce the bug or benchmark the fix.

That release also confirms koboldcpp.net as an official community site; executable downloads remain on the project’s GitHub Releases. Use those linked sources when setting up, rather than a similarly named download site. No software or model download was part of this review.

Next: prove one local scene before adding another frontend

Start with the bundled UI and a small, clearly fictional scene. If you later need a separate character frontend, the existing SillyTavern review explains its connection layer; the Ollama review offers another local-versus-cloud comparison.