Generative image · LoRA fine-tuning · 2026

Style LoRA data ablation on FLUX.2 Klein

Six LoRAs teach FLUX.2 Klein an illustration style I designed, each from a different mix of reference and synthetic images. 252 images scored blind by a VLM judge.

Context: Independent project
Year: 2026
Stack: FLUX.2 Klein 4B, LoRA, AI Toolkit, vLLM, Qwen3.8-27B (judge), NVIDIA GB10

1.49 → 2.92

style score out of 5, base model vs best LoRA

six LoRAs 2.31 to 2.92; scored blind by a VLM judge

LoRAs, one per data mix, same training recipe

47 to 146 training images, 2,000 steps each

89%

of 252 style scores within one point across two judge passes

quadratic kappa 0.73

Base-model image of a fern fox. — Base model

LoRA image of a fern fox in pencil on paper. — Saturnia LoRA 1

Prompt: SATURNIA_STYLE. A slender fox with moth wings, fern ears, and trailing rootlike legs, standing in sparse plants.
Settings: seed 314159 · 28 steps · guidance 4 · 1024x1024

Flat black-line vector fox becomes colored-pencil fur and fern hatching on cream paper; the white ground and hard ink outline are gone.

1 / 10

The Saturnia illustration style was designed by the author, the training data was synthetically generated, and the model was fine-tuned by the author.

Overview

Problem

Image-generation models draw in their own default look. I designed an illustration style, Saturnia: colored pencil and graphite on cream paper. I wanted the FLUX.2 Klein image model to produce it on request, and to find out which training data teaches it best: reference images, synthetic images generated in the style, or a mix.

Approach

Trained six LoRAs on the 4-billion-parameter FLUX.2 Klein. A LoRA is a small add-on that adjusts a large model's output without retraining the whole model.
Each LoRA learned from a different mix of 161 images, 52 reference and 109 synthetic. Everything else was fixed: settings, 2,000 training steps, seeds, and 10% of each source held back.
Generated 36 prompts with the base model and each LoRA at the same seed: 252 images.
Scored them blind. The images were shuffled and given random IDs, and a vision-language model rated each one twice against the project's rubric, using held-back reference images as the style example.

Results

Every LoRA moved images toward the style. Style scores rose from 1.49 out of 5 for the base model to between 2.31 and 2.92, and the judge's two passes agreed within one point on 89% of images. The mixes themselves were mostly indistinguishable. The one clear gap: synthetic images alone scored lower than reference images alone, at about the same dataset size. Prompt following stayed high for every model.

These are a model's ratings, not people's. The judge was lenient on prompt details, each mix was trained once, and smaller mixes saw each image more often.

What I built

Trained six LoRAs on FLUX.2 Klein 4B with one fixed recipe, each on a different mix of 52 reference and 109 synthetic images, to test which data teaches the style.
Scored 252 seed-matched images blind: shuffled random IDs, a local VLM judge applying the project rubric twice, and held-back reference images as the style example.
Every LoRA beat the base model on style (1.49 vs 2.31 to 2.92 of 5), but five of six mixes had overlapping intervals; synthetic-only scored below reference-only.

Results in detail

1Style definition
Saturnia: graphite + colored pencil,
hatching, translucent wash, warm paper
52 reference drawings + captions
2Synthetic dataset generation
Codex image generation, prompted per creature
109 accepted synthetic images in 4 sets
rejects logged in provenance.tsv
3Captioning
sidecar .txt, trigger SATURNIA_STYLE
content-only transform: style sentence dropped
146 train / 15 holdout, deterministic 10% split
4LoRA training on FLUX.2 Klein Base 4B
rank 32, alpha 32, lr 8e-05, 2000 steps, bf16
adamw8bit, batch 1, buckets 512/768/1024
AI Toolkit on NVIDIA GB10 (DGX Spark, 128 GB unified)
5Inference
adapter on the same base model, strength 0.8–1.0
flowmatch sampler, 28 steps, guidance 4.0
matched seed + prompt vs. base for every pair

Figure 1. The pipeline: define the style, build the image set, caption it, train the LoRA on FLUX.2 Klein, then generate images.

Six dataset mixes, one training recipe

Mix	Training images	Train + held out	Passes over data
01 reference only	47 reference	47+5	42.6
02 synthetic core	44 synthetic core	44+5	45.5
03 synthetic core + color	44 core + 22 color	66+7	30.3
04 reference + organic synthetic	47 reference + 66 organic synthetic	113+12	17.7
05 all synthetic	99 synthetic (organic + humanoid)	99+10	20.2
06 everything	47 reference + 99 synthetic	146+15	13.7

Every mix: FLUX.2 Klein Base 4B, LoRA rank 32, alpha 32, learning rate 8e-5, batch 1, 2,000 steps, one training run.

Fixed steps mean smaller mixes repeat each image more often, so data mix and repetition are not separated.

Figure 2. The six dataset mixes. Each was trained with the same recipe for the same number of steps.

Blind rubric scores by dataset mix

Mean of 36 prompts, 1 to 5; whiskers are 95% bootstrap intervals

Style fidelity

base model, no LoRA

1.49

01 reference only

2.92

02 synthetic core

2.31

03 synthetic core + color

2.89

04 reference + organic synthetic

2.69

05 all synthetic

2.72

06 everything

2.86

Generalization

base model, no LoRA

2.10

01 reference only

3.57

02 synthetic core

2.97

03 synthetic core + color

3.35

04 reference + organic synthetic

3.17

05 all synthetic

3.29

06 everything

3.47

Scored blind by a VLM judge (Qwen3.8-27B), not by people. Each image was scored twice; agreement within one point was 89% for style and 88% for generalization.

Held-out Saturnia images scored 3.95 (synthetic) and 4.25 (reference) on style fidelity under the same judge.

Prompt fidelity and artifact control were 4.35 to 4.97 for every model, including base; diversity and base-distinctness were near 5 for all.

Figure 3. Style fidelity and generalization for the base model and the six LoRAs, scored blind by a VLM judge on 36 prompts. Whiskers are 95% bootstrap intervals.

Dataset 06_everything: 146 train + 15 holdout

train
holdout (10% per source)

reference drawings

47 + 5

synthetic core organic

44 + 5

synthetic color organic

22 + 2

synthetic humanoid core

22 + 2

synthetic humanoid color

11 + 1

Figure 4. The largest mix (06 everything) by source, split between images used for training and images held back for checking.

Five beetle images progressing from a small flat render to a pencil drawing. — Figure 5. The same prompt and seed sampled every 500 training steps, from step 0 to step 2,000.

Six frog images; the fourth is a swirl pattern. — Figure 6. Training samples for a prompt asking for bright digital colors. At step 750 the image collapses into a repeating pattern; the final model recovers.

Four images: cartoon ants, three pencil humanoids, a green mantis, a pencil plant-person. — Figure 7. A documented failure: ant and mantis prompts turn into humanoid figures with the LoRA applied.

Grid of twenty creature images in base and LoRA versions. — Figure 8. All ten exported pairs, base model then LoRA, each at the same seed.

Technical details

The full list of measured results behind this project.

Show all 17 measurements

Measurement	Value
dataset-mix ablation design	6 LoRAs, one per data mix (47, 44, 66, 113, 99, 146 training images); identical recipe: rank 32, alpha 32, lr 8e-05, adamw8bit, bf16, batch 1, 2,000 steps, split seed 271828
images scored in the blind rubric evaluation	252 (36 prompts x base + 6 LoRAs, seed 42, 28 steps, guidance 4.0, LoRA strength 0.8), each scored twice by a VLM judge (Qwen3.8-27B) on blinded IDs
style fidelity, 1-5, blind VLM judge, mean of 36 prompts [95% bootstrap CI]	base 1.49 [1.21, 1.81]; 01 reference only 2.92 [2.54, 3.26]; 02 synthetic core 2.31 [1.94, 2.67]; 03 synthetic core + color 2.89 [2.53, 3.24]; 04 reference + organic synthetic 2.69 [2.26, 3.11]; 05 all synthetic 2.72 [2.32, 3.11]; 06 everything 2.86 [2.47, 3.24]
generalization, 1-5, blind VLM judge [95% bootstrap CI]	base 2.10 [1.76, 2.44]; 01 reference only 3.57 [3.28, 3.83]; 02 synthetic core 2.97 [2.57, 3.35]; 03 synthetic core + color 3.35 [3.03, 3.65]; 04 reference + organic synthetic 3.17 [2.75, 3.56]; 05 all synthetic 3.29 [2.90, 3.65]; 06 everything 3.47 [3.11, 3.81]
style fidelity gain over base, paired on prompt	+0.82 to +1.43 points; all six 95% CIs exclude zero (smallest: 02 synthetic core +0.82 [+0.35, +1.28])
prompt fidelity and artifact control, all 7 models	4.35 to 4.97 (base prompt fidelity 4.35 [4.15, 4.53]; base artifact control 4.68 [4.54, 4.82])
judge agreement between the two runs (style fidelity, n=252)	60% exact, 89% within one point, quadratic kappa 0.73
judge positive control: style fidelity of held-out dataset images	synthetic holdout 3.95 (n=20); reference holdout 4.25 (n=4)
passes over the training data at 2,000 steps, batch 1	01 reference only 42.6; 02 synthetic core 45.5; 03 synthetic core + color 30.3; 04 reference + organic synthetic 17.7; 05 all synthetic 20.2; 06 everything 13.7
pairs verified identical seed, prompt, steps, guidance, size, model	30/30 in gallery/everything-lora-radical; 10 exported
regeneration pixel match, fern_fox base (seed 314159)	identical
regeneration pixel match, fern_fox LoRA 1.0 (seed 314159)	identical
generation seconds, 1024px 28 steps, base then LoRA (single run, base includes model load)	158.15 / 69.03
training images (06_everything)	146 train + 15 holdout; 52 reference + 109 synthetic
training wall clock, 2000 steps (from config.yaml and final .safetensors file mtimes)	2.66 h
LoRA checkpoint size	92426856 bytes (rank 32, bf16)
clip duration	9.90 s

Limitations (15)

Rubric scores come from one vision-language judge (Qwen3.8-27B, 4-bit), not from people. No human-rated subset was collected.
The judge is lenient on prompt fidelity: for the beetle-headed gardener prompt no image shows a beetle head, yet 11 of 14 prompt-fidelity scores were 4 or 5. Its style notes also reward creature content, so style and subject are partly mixed.
Each mix was trained once and sampled at one seed (42) per prompt, at LoRA strength 0.8, final checkpoint only. Differences between mixes are not tested across training seeds.
Fixed 2,000 steps means smaller mixes repeat each image more often (13.7 to 45.5 passes), so data mix and repetition are not separated.
Diversity and trigger locality scored near 5 for every model, so they do not separate the mixes. Trigger locality was only half-measured: there are no trigger-free LoRA images.
15 adapter pairs were compared on five criteria without correction for multiple comparisons; a few pairwise differences are expected by chance.
Subject leakage: some non-humanoid prompts (ant_fruit, mantis_fern) become humanoid figures under the LoRA at strength 1.0; see stills/failure-subject-leakage.jpg.
Anatomy drift: leg counts and body plans change between base and LoRA (stag_beetle_branches, walking_bromeliad), so the adapter is not content-preserving.
Intermediate collapse: the step-750 checkpoint turns the vivid-palette prompt into a repeating pattern; the final step-2000 checkpoint recovers.
Palette control: at strength 1.0 the paper ground and pencil texture are imposed even when the prompt asks for a vivid digital look (vivid_bird_frog), and dark-ground prompts (scarab_lotus, firefly_lantern) are moved onto cream paper.
Three of thirty base-model images (dragonfly_reed, caterpillar_bluebells, stick_insect_vine) contain a garbled corner watermark reading roughly 'SATUNNIA'; one training sample (step 0, vivid prompt) has a similar mark. Exported copies were covered with the corner's median color; regions are listed in gallery/index.json and portfolio/build_state.json.
Every exported pair uses one seed (314159); diversity across seeds is not shown here.
Training loss curves are not available for the 4B run: no trainer log was kept for it.
The generation-time metric is one run each; the base number includes first-call model warm-up.
Transparent PNGs were produced by luminance keying, not segmentation; faint paper texture near the subject is kept as low alpha.