LumiCtrl

Learning illuminant prompts for lighting control in personalized text-to-image models. Oral at CVEU, CVPR 2026.

CVPR 2026 Workshop (CVEU) · Oral

Muhammad Atif Butt, Kai Wang, Javier Vazquez-Corral, Joost van de Weijer

arXiv BibTeX

Prompts like “in tungsten light” or “at 6500K” rarely change the illumination of a generated scene in a controlled way. LumiCtrl learns an illuminant prompt from a single image of an object, so that a token such as [c1] reliably renders the scene under one light and [c7] under another, while the subject stays the same.

Change the light
Drag the slider or press play to move through the seven learned illuminant tokens, from warm to cool. Pick a different concept below. All images are results from the paper.

How it works

LumiCtrl has three parts. Physics-based augmentation renders the input under daylight illuminants along the Planckian locus. Edge-guided prompt disentanglement uses a frozen ControlNet so that the learned token captures light rather than structure. A masked reconstruction loss focuses learning on the foreground while letting the background adapt to the new light.