Character LoRA Training, Locally: Free and Uncensored

To train a character LoRA locally you need three things: a GPU with 8–12 GB of VRAM, 15–30 varied images of your character, and a free open-source trainer such as kohya_ss or ostris/ai-toolkit. A first usable LoRA takes roughly one to three hours on a mid-range card and costs nothing.

That is the whole answer; the rest of this guide is the detail. What a character LoRA actually is, how to build a dataset that teaches identity instead of memorizing photos, which of the two standard free trainers to pick, the handful of settings that genuinely matter, and how to test the result instead of hoping. Everything happens on your own machine — no upload, no queue, and no terms of service between you and your own character.

What Is a Character LoRA?

A LoRA (Low-Rank Adaptation) is a small trainable add-on for an image model. Instead of retraining the billions of weights in a base model like SDXL or FLUX, LoRA training adjusts a compact set of extra weights — a file of tens to a couple hundred megabytes — that loads on top of the unchanged base model at generation time. A character LoRA is a LoRA trained on one specific character: the same face, hair, build and signature features across a small set of images, tied to a trigger word you choose. Prompt the trigger word, and the model draws your character.

The problem it solves is consistency. A base model invents a new face every generation; no amount of prompt engineering makes "the same woman as last time" reliable. A character LoRA pins identity in place while leaving everything else promptable — pose, outfit, lighting, scene, art style. One hard constraint to know before training: a LoRA only works with the model family it was trained for. An SDXL character LoRA loads into SDXL-family checkpoints (including finetunes like Juggernaut XL), a FLUX LoRA into FLUX. Train on the family you actually generate with.

The Dataset: 15–30 Images That Teach Identity

How Many, and What Kind

15–30 images is the sweet spot for a single character. Ten can work if they are genuinely varied; going far past fifty rarely improves likeness and mostly teaches the model the repetition in your dataset. What matters more than count is variety with the identity held constant:

• Angles — front, three-quarter, profile; not thirty selfies from the same arm
• Framing — a mix of close-ups, half-body and full-body shots
• Expressions and lighting — neutral, smiling, serious; daylight, indoor, dramatic
• Backgrounds and outfits — different in every image, unless an outfit is part of the character's identity
• One subject per image — no second faces, no watermarks, no heavy filters, nothing you would not want baked into the LoRA

Resolution does not need to be uniform — trainers resize and bucket images automatically — but aim for source material around 1024 px on the long side for SDXL and FLUX. For an original character that only exists as generations, you can bootstrap the dataset from your own renders: generate with a fixed description and seed variations, keep the 20 most consistent results, and accept that the source model's look gets baked in along with the character.

Captioning, Honestly

A caption is a plain text file sitting next to each image, and captioning is where most first LoRAs go wrong, so here is the actual rule: whatever you describe in the caption stays promptable; whatever you leave undescribed gets absorbed into the trigger word. Start every caption with a made-up trigger token the base model has no prior for (an invented name, not a common word), then describe the changeable things — pose, outfit, background, lighting. Do not describe the fixed identity traits you want locked to the trigger, like face shape or eye color. Describing them tells the trainer they are variables, which is the opposite of what you want.

Two honest simplifications save hours. First, many character LoRAs train fine with minimal captions — sometimes just the trigger word — and that is a legitimate strategy, not a hack. Second, auto-captioning gets you 90% of the way: kohya_ss ships BLIP-style caption and WD14 tag generators, and correcting their obvious mistakes beats hand-writing thirty poetic descriptions. Consistency across captions matters more than eloquence in any single one.

Choosing a Local Trainer: kohya_ss vs ai-toolkit

Two free, open-source trainers cover almost all local character training in 2026, and both run entirely on your own machine:

The fair summary: for an SDXL character LoRA the two produce equivalent results, so pick by interface taste — kohya_ss if you want a GUI and the ability to follow ten years of accumulated tutorials knob-for-knob, ai-toolkit if you want a config file, good defaults, and the smoothest road to FLUX training. Neither is one-click software: both want a working Python environment and a CUDA-capable GPU, and the first install is the least plug-and-play step in this entire guide. Budget an evening for setup and treat everything after that as the easy part.

The Settings That Actually Matter

Trainers expose dozens of parameters; a character LoRA cares about five. These are starting points, not gospel — every dataset drifts a little differently:

If the trainer you picked trains the text encoder as well (common for SDXL setups), keep its learning rate lower than the main one — half to a tenth — or leave text-encoder training off entirely for a first run. FLUX LoRAs typically train the transformer only. Resolution: 1024 for SDXL and FLUX, dropping to 768 buckets if an 8 GB card hits out-of-memory errors.

Base Model: SDXL or FLUX?

SDXL is the pragmatic default for character training. It trains on 8–12 GB VRAM with the memory savers above, has the largest ecosystem of compatible checkpoints and style LoRAs, and forgives imperfect datasets. FLUX delivers stronger prompt adherence and detail, but training it realistically wants 16–24 GB of VRAM (some quantized setups squeeze under that, slowly), and the FLUX.1 dev weights carry a non-commercial license worth reading if you plan to sell outputs. If your card is in the 8–12 GB class, train SDXL and enjoy it; the result loads into every SDXL-family checkpoint you already use.

Expect an SDXL character run at these settings to occupy a 12 GB class card for roughly one to three hours. Training holds the whole GPU while it runs — if you normally keep a chat model resident on the same card, the sequential-loading logic from our guide to running an LLM and Stable Diffusion on one GPU applies doubly here: unload everything else first.

Test, Then Iterate

A trained LoRA is a hypothesis until you sweep it. Three tests tell you everything:

• Weight sweep. Fix the seed and prompt, generate at LoRA weight 0.6, 0.8 and 1.0. Too weak and the likeness drifts back toward the base model's generic faces; overtrained and the training data leaks — the same pose and background creeping into every image, plastic skin, cranked contrast.
• Checkpoint sweep. Run the same grid across the checkpoints you saved during training. The best LoRA is very often not the final one; pick the earliest checkpoint with stable likeness, because it is the least overfit.
• Flexibility test. Prompt things that are deliberately not in the dataset: a new outfit, a night scene, a different art style. A good character LoRA follows; one that can only reproduce its training images needs fewer steps, a lower rank, or more dataset variety.

When a first LoRA is 80% right, the fix is almost always the dataset, not the settings. Overtrained results want fewer steps or an earlier checkpoint; a likeness that never locks in wants more varied images; a stubborn background that haunts every render wants that background out of the training set. One evening of iteration on a dataset beats a week of hyperparameter mysticism.

Is Local LoRA Training Uncensored?

Yes, and in a precise, unexciting sense: an open-source trainer running on your own GPU has no content filter, no upload step, and no terms of service. Nothing leaves your machine, so no service reviews your dataset, no policy decides whether your character is acceptable, and no rule change six months from now retroactively applies to you. Hosted training services have to enforce upload rules; a local trainer has nothing to enforce. For adult or NSFW original characters this is the practical difference between possible and not-worth-the-risk — and even for entirely safe-for-work characters it means your designs, your drafts and your reference images stay your own. The same local-first logic drives the chat side of this site, where abliterated models remove refusal behaviour from open-weight LLMs you run yourself.

One honest boundary belongs in this section: uncensored tooling is not a license to train identity LoRAs of real people who never agreed to it. Non-consensual intimate imagery of real people is illegal in a growing list of jurisdictions and indefensible everywhere. Your own character, your own face, or a person who actually said yes — that is the line this guide assumes, and local training does not move it.

Using Your LoRA in Locally Uncensored

This blog belongs to Locally Uncensored, the free, open-source all-in-one local AI studio, so here is the honest hand-off: the app does not train LoRAs — training stays in kohya_ss or ai-toolkit — but it is a comfortable place to use the result. The app drives ComfyUI as its image engine (our ComfyUI beginners guide explains that arrangement), so your finished .safetensors goes where ComfyUI looks for LoRAs: the models/loras folder of your ComfyUI install.

From there, the LoRA picker sits with the Create tab's advanced controls — alongside VAE override and CLIP-skip — out of the way until you ask for it. LoRA stacking is supported with individual weight controls, so your character LoRA can run next to a style LoRA at different strengths, and the built-in CivitAI marketplace downloads any checkpoint, LoRA, embedding or VAE into the correct ComfyUI subdirectory automatically — handy for grabbing style LoRAs to stack with the character you just trained. Type the trigger word, set the weight, generate: same GPU that trained it, no node graph required.

FAQ

How many images do I need for a character LoRA?

15–30 varied images is the sweet spot. As few as 10 can work if they cover different angles, framings, expressions and lighting. Far past 50 rarely helps and mostly teaches the model your dataset's repetition. Variety per image beats raw count.

Can I train a LoRA with 8 GB VRAM?

Yes, with the right base model. SD 1.5 trains comfortably on 8 GB. SDXL fits with every memory saver on — batch size 1, gradient checkpointing, 8-bit optimizer, cached latents — and 768px buckets if you still hit out-of-memory errors. FLUX training is not realistic on 8 GB; plan on 16 GB+, or train SDXL instead.

Is local LoRA training uncensored?

Yes: no content filter, no upload, no terms of service — your dataset never leaves your machine and no service reviews it. The boundary is other people's rights: no identity LoRAs of real people without their consent.

How long does it take to train a character LoRA?

Roughly one to three hours for an SDXL character LoRA at 1,500–3,000 steps on a 12 GB class card. Smaller cards and heavier memory savers run slower, FLUX runs noticeably longer, and your first session includes the one-time cost of setting up the trainer.

Should I train on SDXL or FLUX?

SDXL, unless you have 16–24 GB of VRAM and specifically want FLUX's prompt adherence. SDXL trains on 8–12 GB, has the biggest checkpoint and LoRA ecosystem, and is forgiving. Note the FLUX.1 dev non-commercial license if you sell outputs, and always train on the model family you generate with.

Next Steps

• ComfyUI beginners guide — the engine your LoRA runs on, without the node graphs
• Run an LLM and Stable Diffusion on one GPU — the VRAM logic for sharing a card between chat and images
• Abliterated models guide — the uncensored chat-side counterpart to local image freedom