HEY fellows! I`m Dora, as usual. Lately, one problem that sent me down this rabbit hole was embarrassingly simple: I was making an episodic short-form series for a client, and every time I generated a new clip, the main character looked like a different person. Same prompt, same seed range, same style LoRA — different cheekbones, different eyes, different voice tone. Consistent identity across video clips has been the hardest unsolved problem in AI video generation for a while.
IC-LoRA in LTX 2.3 is the most practical answer I’ve found so far. It’s not magic, but it’s real, it just got proper native ComfyUI support. This guide covers what it actually does, how to train one, and what you should realistically expect from your output.
What IC-LoRA Is and Why It Matters for Video
IC-LoRA stands for In-Context LoRA — a training approach where the LoRA learns to condition generation on reference inputs rather than just text. Standard LoRAs teach the model “what this concept looks like” through captions and training clips. IC-LoRA teaches it something more structured: “given this reference image and this reference audio, produce a video where the subject looks and sounds like the reference.”
The practical payoff for creators: you provide a single reference image (a face, a character, a product) and a short audio clip (a voice sample), and LTX 2.3 generates a video where that person is speaking, moving, and looking consistent with the reference — synchronized audio included. That’s genuinely different from text-prompt consistency tricks, which always drift across generations.
This matters most for anyone building recurring characters: virtual hosts, brand mascots, indie animation, AI-assisted narration series, or product spokespersons. The goal isn’t photorealistic deepfakes — it’s controllable, repeatable identity that holds up across a content pipeline.
IC-LoRA vs Standard LoRA in LTX 2.3
IC-LoRA is a task-specific way to condition video generation on a reference or control signal. The signal depends on the adapter. Do not assume that every IC-LoRA accepts a voice sample or preserves a speaker’s identity.
| Path and exact published release | Reference/control input | Documented job | Boundary |
Ingredients reference-sheet control — Lightricks/LTX-2.3-22b-IC-LoRA-Ingredients, file ltx-2.3-22b-ic-lora-ingredients-0.9.safetensors | One composite reference sheet for the characters, props, and location | Carry those pictured elements into a generated clip | The card specifies 768×448, 121 frames at 24 fps and says other sizes/lengths or omitting the sheet are out of distribution. It does not document voice identity transfer. |
Motion Track Control — Lightricks/LTX-2.3-22b-IC-LoRA-Motion-Track-Control, file ltx-2.3-22b-ic-lora-motion-track-control-ref0.5.safetensors | Reference video with colored spline overlays marking sparse point trajectories | Guide movement of selected objects or regions | This is motion-path control, not a documented face/voice identity adapter. |
Union Control — Lightricks/LTX-2.3-22b-IC-LoRA-Union-Control, file ltx-2.3-22b-ic-lora-union-control-ref0.5.safetensors | Structural control signals | Combine Canny, Depth, and Pose controls | This is structured control, not the ID-LoRA identity/voice path. |
| ID-LoRA 2.3 — ID-LoRA/ID-LoRA, ID-LoRA-2.3 | First-frame image plus short reference audio | The separate project describes identity-conditioned audio-video inference using its released checkpoints | The README describes zero-shot inference with a pre-trained adapter; that is not evidence that every generic IC-LoRA has audio/voice identity features. |
The key distinction: a standard LoRA activates on a text token. IC-LoRA activates a reference input at inference time. You don’t need a trigger word — you hand it a photo and voice clip, and the model does the rest. Before investing in a training setup, compare open-source AI video generators and decide whether inference alone can meet the brief.
Prerequisites (Dataset, VRAM, ltx-trainer Version)
Hardware: An H100 80GB is the officially documented reference. The separate ID-LoRA repository’s README lists Python 3.11+, CUDA 12.x and 24+ GB VRAM as project prerequisites, with 48 GB recommended for two-stage inference. Those are repository setup notes, not a measured training floor or a guarantee that a particular training configuration fits. I found no run log to support an RTX 4090 training claim, an A100 runtime, or a cost estimate, so those figures are not included here.
ltx-trainer version: Use the version from the Lightricks/LTX-2 monorepo, specifically the ltx-trainer package. For the identity-audio variant (ID-LoRA), use the ID-LoRA/ID-LoRA repository which builds on top of ltx-trainer with the audio_ref_only_ic training strategy. As of March 24, 2026, native ComfyUI support for ID-LoRA was merged upstream (PR #13111) — you no longer need a custom node fork.
Model assets required:
LTX-2.3 base checkpoint (~44 GB) → models/checkpoints/
Gemma text encoder (~6 GB) → models/text_encoders/
Spatial upscaler (~700 MB) → models/latent_upscale_models/
Temporal upscaler → models/latent_upscale_models/
Distilled LoRA (~900 MB) → models/loras/Download all assets from HuggingFace before starting. The distilled LoRA is required for the two-stage pipeline that IC-LoRA uses at inference. The official LTX-2 model card on HuggingFace lists all required assets with direct download links and notes that IC-LoRA training in many settings takes under an hour.
Training an IC-LoRA with ltx-trainer
Dataset Requirements for Consistent Identity
This is where IC-LoRA training succeeds or fails. The dataset requirement is more demanding than standard LoRA because you’re teaching a paired relationship — what goes in (reference) must clearly correspond to what comes out (generated video).
Minimum viable dataset:
- 30–50 video clips of the subject
- Each clip: 3–10 seconds, consistent lighting, face clearly visible
- Paired reference frames for each clip — typically the first frame or a held portrait shot
- For voice identity: 5–10 audio reference clips of the subject speaking, each 5–15 seconds
Caption format for IC-LoRA identity training:
[
{
"video_path": "data/clip_001.mp4",
"reference_image": "data/ref_001.jpg",
"reference_audio": "data/voice_001.wav",
"caption": "person speaking directly to camera, natural lighting, slight smile"
},
{
"video_path": "data/clip_002.mp4",
"reference_image": "data/ref_002.jpg",
"reference_audio": "data/voice_002.wav",
"caption": "person walking outdoors, casual clothing, daylight"
}
]Keep captions descriptive but generic — don’t over-specify physical features in the caption text. The reference image carries the identity signal; the caption describes the scene and action. If your captions say “brown-haired woman” but your reference frames show a bald man, the model gets confused about what it’s supposed to learn.
Training Configuration
Clone and set up the ID-LoRA repository for LTX-2.3:
git clone https://github.com/ID-LoRA/ID-LoRA.git
cd ID-LoRA
# Switch to LTX-2.3 workspace
# Edit pyproject.toml: members = ["ID-LoRA-2.3/packages/*"]
uv sync --frozen
source .venv/bin/activateThe training config for identity IC-LoRA (based on the published training_celebvhq.yaml):
model:
checkpoint_path: /path/to/ltx-2.3-22b-dev.safetensors
text_encoder_path: /path/to/gemma-3-12b-it-qat-q4_0-unquantized
dataset:
dataset_file: dataset.json
resolution_buckets:
- "768x432x49"
training_strategy: audio_ref_only_ic # IC-LoRA strategy
optimization:
learning_rate: 1.0e-4
batch_size: 1
max_train_steps: 6000
gradient_checkpointing: true
lora:
rank: 128 # ID-LoRA uses rank 128 for identity fidelity
alpha: 128
validation:
validation_steps: 500
reference_image: /path/to/validation_ref.jpg
reference_audio: /path/to/validation_voice.wav
validation_prompts:
- "person speaking calmly, neutral background"
output:
output_dir: ./outputs/identity_ic_lora_v1Run training:
uv run python ID-LoRA-2.3/packages/ltx-trainer/scripts/train.py \
ID-LoRA-2.3/configs/training_celebvhq.yamlKey difference from standard LoRA: training_strategy: audio_ref_only_ic tells ltx-trainer to use reference conditioning instead of text-only supervision. Rank 128 is significantly higher than standard LoRA (rank 32) — this is what gives IC-LoRA the capacity to encode detailed identity features, but it also means higher memory usage and longer training.
Loading IC-LoRA in ComfyUI Workflow
As the upstream ComfyUI merge (PR #13111), IC-LoRA loading no longer requires a custom node installation. The two nodes you need are now at the core:
LTXICLoRALoaderModelOnly— loads your IC-LoRA.safetensorsand extracts the reference downscale factorLTXAddVideoICLoRAGuide— attaches the reference image/audio as a conditioning guide to the generation pipelineLTXVReferenceAudio— handles reference audio for voice identity transfer
Workflow setup:
[Load LTX-2.3 Checkpoint]
↓
[LTXICLoRALoaderModelOnly] ← ic_lora_weights.safetensors
↓
[LTXAddVideoICLoRAGuide] ← reference_image.jpg + reference_audio.wav
↓
[LTX Sampler / KSampler]
↓
[VAE Decode → Video Output]Copy the trained LoRA to COMFYUI_ROOT/models/loras/ and load it via LTXICLoRALoaderModelOnly. Per the official IC-LoRA workflow documentation on HuggingFace, always use the LTXAddVideoICLoRAGuide node to pass the reference — don’t use the standard LoRA loader, which bypasses the reference conditioning mechanism entirely.
Prompting Tips for Identity Consistency
IC-LoRA changes how you should write prompts. Because identity is carried by the reference input, you don’t need to describe the subject’s appearance in the text prompt — doing so can actually create conflicts.
Do:
"person speaking directly to camera, warm indoor lighting, natural expression"
"subject walking through a park, relaxed pace, late afternoon sunlight"
"presenter gesturing while explaining, clean white background, professional"Avoid:
"young woman with brown curly hair and green eyes speaking..."Describing physical traits in the prompt competes with the reference image signal. The model has two sources telling it what the subject looks like — and they won’t agree perfectly, causing drift or feature blending.
Keep scene and action descriptions concrete. Vague motion descriptors (“moving naturally”) produce inconsistent results. “Slowly turns head left while speaking” gives the model clearer direction. For audio identity, the reference audio clip handles the voice — your text prompt should describe the scene and tone, not vocal characteristics.
Limitations (Drift, Multi-Subject)
Identity drift over long clips: Identity consistency holds well for 5–20 second clips, which maps neatly to LTX 2.3’s generation range. For longer sequences, drift accumulates — the character may look subtly different at second 25 vs second 5. The practical fix is generating in segments and cutting at natural edit points. Whether longer generation windows will maintain identity is still an open research problem.
Multi-subject generation: IC-LoRA is trained on single-identity reference inputs. Two-person scenes with two distinct reference identities are not natively supported in the current ltx-trainer implementation. You can run inference with one reference identity and prompt for a second character, but the second person won’t be reference-conditioned — they’ll be generated from the text description only.
Extreme pose and lighting changes: Reference frames work best when the inference scene’s lighting and camera angle are reasonably similar to training data. Asking an IC-LoRA trained on frontal-face indoor clips to generate a profile-angle outdoor scene will reduce identity fidelity, though it won’t fail completely.
Validation plan — no results reported
No comparison or generation was run for this article. For any future check, use only an image and voice sample you have permission to use. Record the exact model/checkpoint version, full prompt, first-frame image, reference audio, resolution, frame count, frame rate, sampler and guidance settings, seed, run date, hardware, output, and log. Use the same three seeds for each condition and repeat each condition three times. If a workflow cannot accept one of the matched inputs, record that mismatch and describe the comparison as a practical workflow comparison, not a controlled ablation.
| Proposed case | Hold fixed across conditions | Change | Inspect and report |
| ID-LoRA repeatability | ID-LoRA 2.3 checkpoint/version, prompt, first frame, reference audio, all generation settings | Repeat the same condition with three recorded seeds | Face resemblance, exact speech, voice similarity, lip sync, motion defects, and run-to-run variation |
| Practical base-workflow comparison | Same LTX-2.3 base checkpoint/version where available, prompt, first-frame image, reference audio where supported, settings and three seeds | Base workflow versus released ID-LoRA 2.3 workflow | Compare the delivered workflows only. If the base path cannot use speaker-reference audio, this is not a controlled adapter ablation and cannot isolate the adapter’s contribution. |
| Prompt-format check | Variant/checkpoint, reference inputs, output settings, seed set and repeats | For Ingredients, vary only the generated-action text while preserving the reference sheet; for ID-LoRA, vary only one tagged prompt section | Scene adherence and whether the intended character/voice remains recognizable; treat any appearance-description conflict as a hypothesis, not an established effect |
| Pose or lighting change | ID-LoRA checkpoint, prompt, first frame, reference audio, output settings and seed set | Change only one pose, camera, or lighting condition | Identity drift and other visible/audio changes; report outcomes without generalizing to other variants |
| Additional subject | Variant, checkpoint, prompt, settings and consented source material | For ID-LoRA, test a second independently conditioned speaker only if the selected workflow has a documented second reference input | Record whether separate conditioning is available. Ingredients may carry multiple visual elements only when they appear on its reference sheet; do not treat this as multi-speaker voice conditioning |
This plan identifies checks to perform; it does not predict that any adapter will pass them. There is no universal duration or resolution threshold in the evidence reviewed. The Ingredients card was trained at 768×448, 121 frames, 24 fps; the ID-LoRA 2.3 training YAML’s validation dimensions are 512×512×121 at 25 fps, and its one-stage inference example also uses 512×512 and 121 frames. These are variant-specific documented settings, not observed performance or a universal minimum.
FAQ
Q: Can I use an existing face photo I find online as a reference image?
Technically the tool will accept it. But you should not do this for anyone without their consent — the result is a video that makes someone appear to be saying and doing things they didn’t. Beyond the ethical problem, this likely violates platform policies and, depending on jurisdiction, applicable law. Use reference material you own and have consent to use.
Q: Why does my IC-LoRA look fine in validation but drift in full inference？
Validation prompts are short and generated at low resolution — they don’t reveal drift behavior that appears in longer, higher-resolution inference. Check your results at full inference resolution (960×544 or higher) and at the clip length you actually intend to use before declaring the LoRA good.
Q: How many training steps are enough?
The published ID-LoRA configuration uses 6000 steps. Check validation at steps 2000, 3500, and 5000 before completing the run — identity quality often plateaus before 6000, and the optimal checkpoint varies by dataset quality. A clean 35-clip dataset may converge at 4000; a messier 60-clip dataset may need the full 6000 or more.
Previous Posts:
