Run KoboldCpp and LM Studio Together: Several Local LLM Backends in One App
Until 3.0.6, every OpenAI-compatible backend in Locally Uncensored shared one slot. Adding KoboldCpp while LM Studio or the LU Engine was on pushed the other one out onto a standby card (issue 173). In 3.0.7 each backend gets its own row in Settings, AI Backends, with its own name, address, optional key and model IDs, and its models appear in the picker under its name. This guide sets up two servers side by side and goes through what else changed for local servers.
Several backends at the same time, reasoning effort for local models and the chat reply timeout have run in automated tests only, not yet in the app with real servers. The context fixes for KoboldCpp and LM Studio were tested with recorded server answers and a stand in for LM Studio. The think budget ran against a real LU Engine on Windows.
An Example Setup
These are LU's preset addresses for the servers in this guide. The handbook lists all of them.
| Backend | Preset address in LU |
|---|---|
| LU Engine (built in, managed by the app) | 127.0.0.1:8127 |
| LM Studio | localhost:1234 |
| KoboldCpp | localhost:5001 |
| llama.cpp | localhost:8080 |
If your server prints a different port when it starts, use that one. The llama.cpp and LocalAI presets both point at 8080, so if you run both, one of them needs another port, and its row needs the matching address.
Step 1: Add the Second Backend
- Start both servers, for example LM Studio's local server and KoboldCpp with a model loaded
- Open Settings, AI Backends, Providers and click Add Provider
- Pick KoboldCpp from the Local group. With LM Studio already on, KoboldCpp gets a row of its own next to it instead of taking LM Studio's place
- Open the row and check Name and Endpoint. Change the port in Endpoint if your server uses another one
- Click Test
A server with no preset goes in through Custom (OpenAI-compat), which takes any address that speaks the OpenAI API.
Two shortcuts exist. When LU finds several local backends at startup, its dialog reads "Multiple backends running. Select your primary backend, or add all of them side by side, each with its own models." and has a button Add all side by side. And a backend that an earlier version pushed onto the standby card now has a Run side by side button there. Chats and settings from before keep their models.
Step 2: Pick Models From Both
The model picker lists each backend's models under its name. A group chat can mix models from several backends, so a model in LM Studio and one in KoboldCpp can answer in the same chat.
A server that serves a model without listing it gets a new field on its provider card: Model IDs the server does not list (issue 112). Write one ID per line, up to 20. They join that server's models in the picker while the server is running.
Step 3: Check the Context Window
For a model of such a backend, the context window comes from that backend's own server (KoboldCpp, llama.cpp, LM Studio). So do the history budget of a chat, the room for an attached file and the extra samplers. A server on your network is never cut to the send limit of a paid provider.
KoboldCpp no longer stuck at 4K
KoboldCpp behind its Ollama API sends no context length. LU used to take 4096 as the model's maximum, so the context picker offered only 4K, and a long chat was cut to fit 4K while a much larger window from Settings went out with each message (issue 166). LU now also asks KoboldCpp itself and the list of loaded models. When nobody names a length, the picker offers every size and says: "This server did not report the model's context length, so LU cannot check these sizes. Pick the size the server runs with." The history is now trimmed to the window that is really sent. This was tested with recorded server answers, not yet against a running KoboldCpp.
LM Studio keeps your size
A size you pick in the context picker is now saved for that model and used whenever LU loads it in LM Studio, also after you switch to another model and back. It no longer changes the Ollama context window. A size picked before this update needs to be picked once more. Making another model the main model no longer unloads a model that still speaks in the group chat on screen (issue 168). We tested this with a stand in for LM Studio, not yet with LM Studio itself.
Step 4: Reasoning Effort on Local Models
The effort button next to Think cycles through Low, Medium, High and Max. It used to work for hosted models only. Now local models have it in the form each server reads (issue 176):
| Server | What it gets |
|---|---|
| LU Engine | The level in the model's chat template as reasoning_effort. Low and Medium also set a think budget of 1024 and 4096 tokens unless you set one yourself; High sets none |
| LM Studio 0.4.8 and later | reasoning_effort with the levels LM Studio lists for each model |
| KoboldCpp | reasoning_effort for Low, Medium and High. Max sends nothing, which KoboldCpp runs without a limit |
| Your own llama.cpp server | The level in the chat template, when the model's template reads it |
Everywhere else the button is greyed and says why, for example "LM Studio lists no effort levels for this model." or "This backend does not take effort levels." Think off still means no thinking. The effort level is one setting for all models, not one per model, and there is no separate None level. This has run in automated tests only, not against a real LM Studio, KoboldCpp or llama.cpp server.
Think budget
Think budget sits next to Think in the sampling popup and in Settings, General, Generation. It sets how many tokens a model may spend on thinking, and 0 keeps the automatic choice. Only the LU Engine and Anthropic take it. For LM Studio, KoboldCpp, Ollama and other OpenAI-compatible servers the field is off and says so.
Step 5: Give Long Prompts Time
A local chat used to stop with "the backend accepted the request but sent nothing for 600 s", even while the backend was still reading a very long prompt (issue 177). Settings, General now has a section Chat Reply Timeout: "Seconds a chat waits for the first word of an answer from a local backend before LU gives up." The field takes 30 seconds up to 24 hours, and the Off box waits as long as the connection stays open. The default stays 600 seconds, and Stop always ends the wait. LU does not show the progress of the prompt reading yet. This has run in automated tests only.
Limits
- Several backends at the same time, reasoning effort and the chat reply timeout have not run against real LM Studio, KoboldCpp or llama.cpp servers yet
- The KoboldCpp context fix was tested with recorded answers, the LM Studio one with a stand in
- The think budget works with the LU Engine and Anthropic only
- Settings, Voice & Remote, Local API, the address LU gives other programs, still reaches only the LU Engine, Ollama and LM Studio
Questions
Can I run KoboldCpp and LM Studio at the same time?
Yes. Since 3.0.7 each OpenAI-compatible backend gets its own row in Settings, AI Backends, with its models under its name in the picker. This has run in automated tests, not yet with two real servers.
My server serves a model it does not list. How do I use it?
Write its ID into Model IDs the server does not list on that server's provider card, one per line. It joins the picker while the server is running.
Why did KoboldCpp only offer 4K context?
KoboldCpp behind its Ollama API sends no context length, and LU took 4096 as the maximum. LU now asks KoboldCpp itself and the list of loaded models, and if nobody names a length, the picker offers every size.
The chat stops while my server reads a long prompt. What can I do?
Raise the Chat Reply Timeout under Settings, General, up to 24 hours, or tick Off. The default is 600 seconds.
Related
- Locally Uncensored vs KoboldCpp
- Locally Uncensored vs LM Studio, the comparison page
Locally Uncensored is AGPL-3.0 licensed and built by PurpleDoubleD. Corrections welcome on GitHub Discussions or in the Discord.
In the handbook
- Settings and troubleshooting, the Providers list with every preset address, and the Local API
- Chat, the model picker, the context window, Think and effort
- First start, the backend scan and the ports it checks