[ICML 2026] Motion Attribution for Video Generation
Authors: Xindi Wu, Despoina Paschalidou, Jun Gao, Antonio Torralba, Laura Leal-Taixé, Olga Russakovsky, Sanja Fidler, Jonathan Lorraine
Paper: https://arxiv.org/abs/2601.08828, ICML Submission
Project site: https://research.nvidia.com/labs/sil/projects/MOTIVE/
Code: N/A
Model: N/A
TL;DR
WHAT was done? The authors introduce MOTIVE (MOTIon attribution for Video gEneration), a motion-centric, gradient-based data attribution framework designed for modern, large-scale video diffusion and flow-matching models. It successfully isolates and attributes temporal dynamics rather than static visual appearance by computing gradients through motion-weighted loss masks, utilizing structured low-rank projections and frame-length normalization to scale to billion-parameter backbones.
WHY it matters? Current video generative models scale through massive web scraping, yet we lack diagnostic tools to understand which training clips teach models physical mechanics like bouncing, rolling, or fluid dynamics. Developed by researchers at NVIDIA, Princeton, and MIT, MOTIVE isolates motion-specific gradients to enable targeted, data-centric curation. Fine-tuning on just 10% of MOTIVE-selected data matches or outperforms full-dataset training, demonstrating a highly efficient path toward more physically consistent world models.
This paper has the ICML 2026 Outstanding Paper Honorable Mention.
Details
The Kinetic Blindspot in Generative Data Curation
Modern video generative models have achieved spectacular visual fidelity, yet they remain notoriously fragile when simulating physical forces, often suffering from temporal flicker, identity drift, and physically implausible motion trajectories. While researchers have historically addressed these limitations through architectural alterations or reinforcement learning, the quality and composition of the underlying training data remains the most critical lever. Unfortunately, identifying which training videos shape specific motion patterns is incredibly difficult. Prior gradient-based data attribution methods, designed primarily for static images, collapse the temporal axis when applied to video, conflating dynamic behavior with static appearance such as backgrounds, textures, and objects. Consequently, a model trying to analyze “floating” might erroneously attribute its knowledge to videos sharing a similar blue water background rather than the actual physical interaction of buoyancy. To bridge this gap, a framework must explicitly isolate and measure temporal dynamics instead of treating time as an additional spatial dimension.
Defining Motion Saliency: The Flow-Based Latent Grid
To establish a motion-centric attribution framework, we must first formalize how motion is mathematically represented and isolated from static appearance. Let a video clip be denoted as v=[ff]f=1F, where ff∈RH×W×3 is the f-th frame in a sequence of length F, height H, and width W. Motion is represented via dense optical flow between consecutive frames: Ff:{1,…,H}×{1,…,W}→R2, where each flow vector encodes the horizontal displacement dw and vertical displacement dh of a pixel. The motion magnitude at any location is then defined as Mf(h,w)=∥Ff(h,w)∥2. To ensure that different videos can be compared regardless of their absolute motion scale, these magnitudes are min-max normalized to produce a normalized motion weight map W(f,h,w)∈[0,1]:
In this formulation, mmin and mmax represent the minimum and maximum motion magnitudes observed across all frames and pixels, while ζ=10−6 is a small numerical stabilizer to prevent division by zero. Because modern diffusion networks operate within compressed latent spaces, these normalized motion weights must be bilinearly downsampled to the latent spatial grid index (h̃, w̃) corresponding to the latent video representation h=E(v)∈RF×H/s×W/s×C, where s=8 is the downsampling factor and C=16 is the channel dimension of the encoder. This yields the latent-aligned motion weights W̃v,c(f,h,w̃), which serve as the spatial-temporal masks that isolate dynamics from background textures.
The MOTIVE Pipeline: From Pixel Trajectories to Gradient Masking
The execution flow of MOTIVE, illustrated in Figure 1, processes a single training video and a target query video to evaluate their mutual kinetic influence.
Consider a query video v̂, such as a foam cube bobbing in water. First, the dense point tracker AllTracker extracts the per-pixel flow displacements Df(h,w)=(dw,dh), which are converted to motion magnitudes and bilinearly downsampled to align with the latent grid. This downsampled mask, visualized in Figure 9 where static backgrounds are attenuated to neutral gray, is multiplied by the per-location squared error in the latent space.
The primary motion-weighted loss function is formalized as:
Here, Fv represents the frame length of the video clip, θ denotes the trainable parameters of the Diffusion Transformer (DiT) backbone, and c represents the text conditioning signals. The per-location loss L̃θ,v,c is defined at a fixed timestep tfix and fixed noise draw ϵfix as:
During backpropagation, this loss-space masking forces the network to only compute gradients for the parameters that influence the dynamic regions of the video. The resulting raw gradient gmot=∇θLmot is normalized to correct for frame-length biases and projected into a highly compressed, lower-dimensional space. The same sequence is executed for any candidate fine-tuning sample vn. Finally, the motion-aware influence score Imot(vn,v̂) is computed as the cosine similarity between their normalized projected gradients:
where the projected, normalized gradient is defined as:
In this formulation, P∈RD′×D represents a structured random projection matrix, allowing the similarity of high-dimensional gradients to be computed in a lightweight space.
Engineering Scalability: Dimensional Compression and Length Normalization
Scaling gradient-based data attribution to modern billion-parameter video generators like Wan2.1 or LTX-Video introduces massive storage and computational bottlenecks. Standard gradient attribution requires storing the full gradient vector of size D≈1.4×109 for every training sample, which is completely intractable. To resolve this, MOTIVE utilizes a structured Johnson-Lindenstrauss projection matrix P implemented via Fastfood. This reduces the gradient dimensionality from D to a highly compact D′=512. As demonstrated in the projection dimension analysis in Figure 5, a dimension of D′=512 strikes an optimal trade-off between accuracy and efficiency, achieving a Spearman rank correlation of ρ=74.7% compared to unprojected gradients, while keeping the projection cost at a negligible O(D′logD′) per example.
Furthermore, video datasets often exhibit significant variation in sequence lengths. Raw gradient magnitudes naturally scale with the number of frames F, which creates a severe duration bias where longer clips appear misleadingly influential regardless of their actual motion relevance. Without correction, gradient-based rankings correlate with video length at ρ=78.0%, heavily favoring longer files. MOTIVE mitigates this by normalizing the raw gradient by 1/F before the projection step. As shown in Figure 4, this frame-length normalization successfully eliminates length bias, dropping spurious correlation by 54.0% and ensuring that the selected training clips exhibit coherent, target-aligned motion dynamics rather than simply being the longest files in the dataset.
Quantitative Breakthroughs: Outperforming Full-Dataset Tuning
The validation of MOTIVE spans multiple large-scale video backbones, including Wan2.1-T2V-1.3B, Wan2.2-TI2V-5B, and LTX-2B, evaluated on the VIDGEN-1M and 4DNeX-10M datasets. Quantitative results on VBench, detailed in Table 1 and Table 5, demonstrate that MOTIVE-guided data selection (utilizing only 10% of the training dataset) consistently outperforms random selection and whole-video attribution. On the Wan2.1 model, MOTIVE achieves a VBench dynamic degree score of 47.6%, vastly outperforming random selection (41.3%) and the unmasked baseline (43.8%). Remarkably, fine-tuning on this 10% MOTIVE-selected subset even surpasses full fine-tuning on 100% of the dataset, which only achieves a dynamic degree score of 42.0%.
To ensure that these improvements represent actual physical fidelity rather than simply picking “motion-rich” clips, the authors perform a distribution analysis of the selected data’s motion magnitude, shown in Figure 6. The mean motion magnitude of MOTIVE’s top 10% selected clips is 3.85, which is only 4.3% higher than the bottom 10% (3.69). This demonstrates that MOTIVE is selecting highly precise, structurally influential motion patterns rather than acting as a simple motion-saliency filter.
Qualitative comparisons in Figure 3 and human evaluations in Table 2 confirm this, with annotators favoring MOTIVE with a 74.1% win rate against the base model and a 53.1% win rate against full-dataset fine-tuning.
Placing MOTIVE in the Data Attribution Landscape
MOTIVE builds directly upon the mathematical foundations of influence functions originally formalized by Koh & Liang. To make influence calculation feasible at modern network scales, it adapts gradient similarity approximations from TracIn and TRAK. To handle the timestep-dependent biases unique to diffusion models, MOTIVE extends the single-timestep, variance-reduced heuristic of Diffusion ReTrac by fixing the evaluation at tfix=751, which corresponds to the critical mid-denoising stage where structural motion trajectories are formed. While prior extensions of data attribution to diffusion models, such as Concept-TRAK, focus on semantic and static visual factors, MOTIVE represents a critical departure by being the first framework designed specifically to isolate and attribute temporal dynamics across frames.
Critical Vulnerabilities: Compute Bottlenecks and Ego-Motion Distortions
Despite its impressive performance, MOTIVE has several clear limitations. First, the upfront computational cost remains very high. As detailed in the runtime breakdown in Table 6, computing the per-training-sample gradients for a modest set of 10,000 clips requires roughly 150 hours on a single NVIDIA A100 GPU. While this cost is a one-time overhead that is amortized across all subsequent queries, it presents a steep barrier to scaling the pipeline to web-scale pre-training datasets.
Second, because MOTIVE averages the motion-weighted loss across the entire duration of a video, it risks diluting highly informative but short-duration motion segments. If a long video consists of mostly static frames with only a fraction of a second of relevant motion, the average gradient signal will be heavily diluted. Furthermore, the current optical-flow-based masking can be easily corrupted by camera ego-motion. While MOTIVE implements a heuristic to down-weight spatially uniform flow fields, a mathematically rigorous disentanglement of camera movement from object motion remains an open challenge.
Strategic Verdict: A Data-Centric Future for World Modeling
MOTIVE represents a massive milestone in the transition from brute-force scale to highly curated, data-centric AI. By proving that video diffusion models can learn complex physics, deformations, and temporal dynamics more efficiently with just 10% of highly targeted data, the paper challenges the “more-is-always-better” data scaling paradigm. For researchers building physical AI, robotic simulations, and world models, MOTIVE provides an invaluable diagnostic and curation toolkit. The ability to mathematically trace physical behaviors back to individual training instances is a crucial step toward building more controllable, physically consistent, and safer generative world models.