· 2 min read
Shipping image generation: from prompt to on-brand image
Turning a diffusion model into a product clients can rely on: prompt templates, reference images, brand styles with LoRA, reproducible seeds, and a GPU job queue that doesn't fall over.
Jose Rai
Senior software engineer
Generating one good image in a notebook is easy. Building something a client's team uses every day, with on-brand results, predictable cost and images you can recreate next month, is a different job. I've built image-generation products for private clients, and the hard parts were almost never the model.
Don't expose the raw prompt
Users write short prompts like "summer sale banner, beach". Diffusion models do best with detailed ones. So the app wraps what the user types in a template per use case, with the brand's style terms, lighting, composition and a shared negative prompt, and the user only picks options that matter to them: format, mood, product. Results become consistent, and the template is the one place to improve quality for everyone.
References and brand style
- Reference images (image-to-image, or ControlNet for pose and layout) keep a product's shape or a layout fixed while the style changes.
- A LoRA trained on the brand's own images gets their look far more reliably than describing it in words. It's small, cheap to train, and loaded per client.
- Fixed sizes per format. Generating at the model's native resolution and upscaling afterwards gives cleaner results than asking for odd sizes directly.
Make every image reproducible
Every job stores its full recipe: model version, LoRA, template, final prompt, negative prompt, seed, steps and guidance. When a client loves an image and wants five variations next month, the same seed and settings bring it back, and small changes to the prompt give controlled variations instead of a lottery.
import torch
from diffusers import StableDiffusionXLPipeline
pipe = StableDiffusionXLPipeline.from_pretrained(
"stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16
).to("cuda")
pipe.load_lora_weights("loras/client-brand.safetensors") # loaded per client
def generate(job):
# The job row holds the whole recipe, so any image can be recreated later.
generator = torch.Generator("cuda").manual_seed(job.seed)
image = pipe(
prompt=job.prompt,
negative_prompt=job.negative_prompt,
width=job.width,
height=job.height,
num_inference_steps=job.steps,
guidance_scale=job.guidance,
generator=generator,
).images[0]
return imageA queue between the app and the GPUs
Generation takes seconds to minutes, and GPUs are the expensive part, so the web app never calls the model directly. The Next.js app creates a job through the API, a queue hands it to a GPU worker, the worker uploads results to object storage, and the app shows progress until the images arrive. Workers keep models loaded in memory between jobs, which matters more for speed than almost any setting.
- Limits per user and per client, so one batch of 200 variations doesn't starve everyone else.
- Safety checks on prompts before the job and on images after it, with the client's own banned terms added on top.
- Cost per image tracked from day one. GPU seconds per job, multiplied out, is the number that decides pricing.
Clients don't buy a model; they buy images they can use. Templates for consistency, a brand LoRA for identity, recipes for reproducibility and a calm job queue are what turn a diffusion model into that.