Houdini

We’ve all seen the impressive AI video clips floating around lately. By combining a First Frame (FF) image with a Reference Video, creators are getting incredibly believable outputs, especially when the reference footage relies on simple, solid geometries.

But believability isn’t the same thing as physical accuracy.

As AI models get better at mimicking motion, a major question remains: How much can we trust AI to properly handle physics-based rendering, like complex light transmission, refraction, and dynamic shadows?

To find out, I set up a strictly controlled environment to test whether modern AI video generators actually “understand” light, or if they just hallucinate pretty pixels.

The Setup: Establishing the Ground Truth

To avoid any ambiguity, I bypassed real-world camera footage entirely. Instead, I built a control test completely in Houdini, complete with dialed-in materials, physically accurate lighting, and a moving light source.

The Scene: A bright, spherical light source translating horizontally behind a transparent orange cube, casting dynamic soft, colored shadows. The Inputs for AI:

• Reference Video: The raw Houdini Flipbook viewport preview (to provide the exact geometric motion path).
• First Frame: The first frame of the fully rendered and denoised Houdini sequence.
• Prompt: “Moving bright ball of light. Realistic shadow cast by the transparent orange cube. Use reference image as first frame.”

I ran these identical inputs through two of the leading multi-modal video generators: Kling 3.0 and Seedance 2.0. Here is the breakdown of the results.

The Results Analysis

1. The Ground Truth (Houdini + Denoise)

Before judging the AI, let’s look at how actual physics behaves. In the Houdini render, as the bright sphere moves from left to right behind the transparent cube, the resulting colored shadow shifts its angle in exact, inverse correlation to the light source. The orange tinted shadow projected onto the ground is physically accurate and scales logically as the light source changes position.

2. Kling 3.0: Aesthetically Pleasing, Physically Almost There

Kling 3.0 does a highly commendable job of maintaining the initial aesthetic established by the first frame. The texture of the ground and the glass-like quality of the cube remain quite stable, and the overall movement of the ball and the reflections off the cube are remarkably similar to the control footage.

The Physics Breakdown: Kling actually does a surprisingly decent job mimicking shadow-casting logic for a majority of the clip. It manages to infer depth; when the ball travels further to the right of the frame, Kling correctly elongates the shadow, matching the behavior of the Houdini control footage. However, its understanding of absolute spatial coordinates breaks down at specific points in time. As seen in the reference frame below, when the ball is directly behind the cube, Kling fails the geometry test and skews the shadow to the left, rather than casting it directly in front of the cube. It is approximating the physics based on visual context, rather than calculating actual spatial relationships.

3. Seedance 2.0: Loose Tracking and Creative Liberties

Seedance 2.0 seems to have a mind and opinion of its own. Despite being fed the same prompts and reference video, it actively decided to ignore the 1:1 motion constraints and initiated a camera pan instead.

The Physics Breakdown: While it abandoned the tracking data (the reference video clearly shows the relative size and position of the ball relative to the cube, yet Seedance interprets the ball’s movement as perfectly symmetrical from left to right), the execution of its own idea is impressive. It handles the camera pan smoothly, and the lighting and material rendering look completely comparable to both Kling and the Houdini control.

The Verdict

The true state of AI video generation isn’t that it completely fails at physics, but rather that it relies on highly educated, aesthetic approximation instead of mathematical simulation.

Kling 3.0 proves that AI can infer complex physical relationships, like elongating a shadow as a light source moves further away, but it will still slip up on basic spatial alignment when it lacks a true 3D coordinate system. Seedance 2.0 proves that these models can render beautiful, accurate materials and lighting, but they often prioritize cinematic flair over strict adherence to your control inputs.

Ultimately, if you do not need exact 1:1 motion tracking with your control footage, tools like Seedance successfully fail the mission by providing gorgeous, usable alternatives. But if you are working in a pipeline that demands absolute precision, where a light must accurately cast a shadow directly beneath it, AI is still a black box of creative liberties. For true control, procedural workflows remain undefeated.

As a visual effects artist, procedural fluid simulations are a staple in my workflow. Building custom velocity fields, tweaking flip solvers in Houdini, and dialing in physically accurate rendering for water takes time, computational power, and a lot of patience. With the rapid explosion of AI video-to-video models, a natural question arises: can these tools bypass the heavy lifting of traditional fluid sims?

Specifically, I wanted to test whether AI can accurately interpret water movement, the refraction of light through a volume, and water caustics. To find out, I set up a stress test comparing a traditional procedural workflow against two AI models: Seedance 2.0 and Kling 3.0.

Here is a breakdown of the experiment and the results.

The Control: Procedural Accuracy in Houdini

To establish a baseline, I built a controlled simulation in Houdini. The setup was straightforward but designed to test specific optical properties:

• A small ocean geometry generating surface waves, placed over a simple grid.
• A yellow cube placed on the grid, fully submerged under the water.
• Materials accurately assigned to calculate the Index of Refraction (IOR).
• Caustics explicitly turned on and lighting set to highlight the light patterns on the floor.

The scene was rendered in Karma. The very first frame of this render served as the ground truth (and also reference image) for how light should bend around the submerged cube and cast caustics on the ocean floor. The whole 3-seconds shot was also rendered out and composed for comparison with other Gen AI outputs.

The Methodology

To test the AI, I provided the models with the exact same starting point and motion data.

1. The Input Frame: The extracted first frame from the Houdini render.
2. The Reference Motion: An extracted flipbook animation (mp4) of the waves and the motion of the small ocean geometry.
3. The Prompt: Use ff as first frame. ff shows a yellow cube in shallow water with water caustics, distorted by refraction of light. Use Video1 as reference video for movement of water and position of cube. Interpret the distortion of the cube and water caustics. No audio.

This exact combination was fed into both Seedance 2.0 and Kling 3.0.

The Results: How Did the AI Perform?

Evaluating the outputs required looking past the initial “wow” factor of AI generation and critically analyzing the physics of the scene.

Seedance 2.0: Good Motion, Broken Physics

• Water Movement: Seedance did a surprisingly good job imitating the motion of the waves from the flipbook reference. It definitely looked like a fluid, and the pure white reflections scattered across the surface were convincing.
• Material Properties: This is where the physics fell apart. Instead of rendering clear water in a pool, Seedance generated what looked like an opaque, sky-blue liquid.
• Refraction: Because the model hallucinated a highly opaque liquid, the physics of light transport were completely illogical. A thick, opaque liquid shouldn’t allow light rays from a submerged cube to refract that clearly. Setting that aside, the distortion on the cube was extremely minimal compared to the Karma render. It appeared the model simply morphed the object toward the yellow square I left in the reference video for general positioning. (Note for future tests: removing the tracking square might give the AI more freedom to interpret genuine optical distortion rather than acting as a rigid morph target).
• Water Caustics: Due to the opaque nature of the generated fluid, the caustics were almost entirely non-existent.

Kling 3.0: Total Hallucination

• Output: The result from Kling 3.0 was completely unusable for this specific use case. The model entirely misinterpreted the prompt and reference, generating what looked like a frozen layer of ice with water running underneath it. While there were some vague caustic-like patterns on the pool floor and distortion of the cube, the overall output was a failed interpretation of the input data.

Conclusion

While video-to-video AI models are making incredible strides in tracking 2D surface displacement, this test highlights a massive gap in their ability to understand volumetric data and complex physical light transport.

The AI tools treated the reference as a 2D warping task. Seedance 2.0 successfully mimicked the surface fluidity but failed to comprehend the depth of the water, fundamentally breaking the material properties by turning a clear, refractive medium into an opaque liquid. Kling 3.0 simply hallucinated entirely different physical states.

For now, if a project requires precise optical fidelity, where refraction and caustics need to accurately interact with submerged objects, it still absolutely demands a robust procedural solver and a physically based renderer. AI can approximate the broad strokes of movement, but true fluid physics remains securely in the realm of traditional 3D pipelines.