Seedance – Tan Zi Jian

July 8, 2026

AI vs. Physics: Can Video-to-Video Models Handle Fluid Refraction and Caustics?

As a visual effects artist, procedural fluid simulations are a staple in my workflow. Building custom velocity fields, tweaking flip solvers in Houdini, and dialing in physically accurate rendering for water takes time, computational power, and a lot of patience. With the rapid explosion of AI video-to-video models, a natural question arises: can these tools bypass the heavy lifting of traditional fluid sims?

Specifically, I wanted to test whether AI can accurately interpret water movement, the refraction of light through a volume, and water caustics. To find out, I set up a stress test comparing a traditional procedural workflow against two AI models: Seedance 2.0 and Kling 3.0.

Here is a breakdown of the experiment and the results.

The Control: Procedural Accuracy in Houdini

To establish a baseline, I built a controlled simulation in Houdini. The setup was straightforward but designed to test specific optical properties:

A small ocean geometry generating surface waves, placed over a simple grid.
A yellow cube placed on the grid, fully submerged under the water.
Materials accurately assigned to calculate the Index of Refraction (IOR).
Caustics explicitly turned on and lighting set to highlight the light patterns on the floor.

The scene was rendered in Karma. The very first frame of this render served as the ground truth (and also reference image) for how light should bend around the submerged cube and cast caustics on the ocean floor. The whole 3-seconds shot was also rendered out and composed for comparison with other Gen AI outputs.

The final shot that was made using Houdini and composed in After Effects.

The Methodology

To test the AI, I provided the models with the exact same starting point and motion data.

The Input Frame: The extracted first frame from the Houdini render.
The Reference Motion: An extracted flipbook animation (mp4) of the waves and the motion of the small ocean geometry.
The Prompt: Use ff as first frame. ff shows a yellow cube in shallow water with water caustics, distorted by refraction of light. Use Video1 as reference video for movement of water and position of cube. Interpret the distortion of the cube and water caustics. No audio.

The Flipbook render from Houdini used as reference video

This exact combination was fed into both Seedance 2.0 and Kling 3.0.

The Results: How Did the AI Perform?

Evaluating the outputs required looking past the initial “wow” factor of AI generation and critically analyzing the physics of the scene.

Seedance 2.0: Good Motion, Broken Physics

Seedance 2.0 output

Water Movement: Seedance did a surprisingly good job imitating the motion of the waves from the flipbook reference. It definitely looked like a fluid, and the pure white reflections scattered across the surface were convincing.
Material Properties: This is where the physics fell apart. Instead of rendering clear water in a pool, Seedance generated what looked like an opaque, sky-blue liquid.
Refraction: Because the model hallucinated a highly opaque liquid, the physics of light transport were completely illogical. A thick, opaque liquid shouldn’t allow light rays from a submerged cube to refract that clearly. Setting that aside, the distortion on the cube was extremely minimal compared to the Karma render. It appeared the model simply morphed the object toward the yellow square I left in the reference video for general positioning. (Note for future tests: removing the tracking square might give the AI more freedom to interpret genuine optical distortion rather than acting as a rigid morph target).
Water Caustics: Due to the opaque nature of the generated fluid, the caustics were almost entirely non-existent.

Kling 3.0: Total Hallucination

Kling 3.0 output

Output: The result from Kling 3.0 was completely unusable for this specific use case. The model entirely misinterpreted the prompt and reference, generating what looked like a frozen layer of ice with water running underneath it. While there were some vague caustic-like patterns on the pool floor and distortion of the cube, the overall output was a failed interpretation of the input data.

Conclusion

While video-to-video AI models are making incredible strides in tracking 2D surface displacement, this test highlights a massive gap in their ability to understand volumetric data and complex physical light transport.

The AI tools treated the reference as a 2D warping task. Seedance 2.0 successfully mimicked the surface fluidity but failed to comprehend the depth of the water, fundamentally breaking the material properties by turning a clear, refractive medium into an opaque liquid. Kling 3.0 simply hallucinated entirely different physical states.

Feature Tested	Seedance 2.0	Kling 3.0
Water Movement	Convincing surface tracking and fluid motion.	Unusable (hallucinated an ice layer).
Material Accuracy	Failed (rendered an opaque liquid).	Failed.
Refraction	Minimal; hindered by opacity and reference morphing.	Some distortion present but unnatural-looking.
Caustics	Barely visible due to incorrect material density.	Failed.

For now, if a project requires precise optical fidelity, where refraction and caustics need to accurately interact with submerged objects, it still absolutely demands a robust procedural solver and a physically based renderer. AI can approximate the broad strokes of movement, but true fluid physics remains securely in the realm of traditional 3D pipelines.

Generative AI Kling Seedance

Zi Jian

June 18, 2026

The Last-Minute PPL Pivot: Swapping a Car in a Drone Shot with AI & Compositing

So, here’s a classic indie film scenario. You finally lock picture on a scene, everything is tracking nicely, and the director is happy. Then the producer walks into the post suite with a massive grin because they just closed a last-minute Product Placement (PPL) deal with BYD. The catch? The car we actually shot on location during a smooth, 5-second aerial drone sequence is a bright blue SUV. Now, it absolutely has to be a white BYD car.

The 5-second aerial drone sequence to be worked on.

Instead of booking an expensive reshoot or spending days on a full 3D asset tracking pipeline, I wanted to see if I could solve this using a generative AI workflow while keeping the original Full HD plate completely crisp.

Here’s how the process went, the roadblocks I hit, and how I blended AI with traditional compositing to make the shot work.

Avoiding the “AI Hallucination” Trap

When you’re doing a specific vehicle swap for a brand, your biggest hurdle is fidelity. If the AI shifts the contours or gives it a weird generic shape, the illusion falls apart instantly. Since newer EV models like BYD aren’t heavily represented in standard base AI models, my biggest concern was making sure the model wouldn’t hallucinate a totally different car.

I figured the easiest path would be a First Frame + Video-to-Video workflow.

First Frame + Seedance 2.0

I pulled the first frame of the drone shot, grabbed a reference photo of the white BYD car from the internet, and used Nano Banana Pro to generate a clean, structurally accurate first frame.

Then I tossed that generation into Seedance 2.0 for the video-to-video pass. The car reproduction was solid, and it actually nailed the layout of the wheels. However, the environment continuity completely broke down. After a few frames, the background would suddenly jump out of alignment, making the footage completely unusable for a continuous camera move.

Output produced by Seedance 2.0.

Switching to Kling

I ran the exact same first-frame setup through Kling instead. Kling handled the camera tracking perfectly, and the spatial continuity matched the smooth, sweeping drone move exactly.

The downside was the overall image quality of the output. The original footage had a lot of motion blur from the drone’s speed, and the model struggled to upscale those fine details cleanly, leaving the plate looking pretty muddy.

Output produced by Kling.

Cutting Out the AI Noise

At this point, you don’t just throw your hands up; you use traditional VFX logic. If the AI environment looks messy but the car tracking is spot-on, you isolate the car and throw away the rest of the AI noise.

To keep the original Full HD drone plate completely clean, I built a quick compositing bridge. I rotoscoped just the white BYD car out of the muddy Kling render and added a subtle feather to the mask edges so it would transition naturally into the asphalt.

Then, I dropped the isolated AI vehicle directly on top of the original blue SUV on the untouched Full HD plate. To lock it into the scene, I matched the black levels, highlights, and color temperature of the white car to the ambient lighting of the original plate.

Output after compositing Kling 3.0’s car output over the original footage

Final Thoughts

The final composite looks completely convincing in motion. If I’m nitpicking, the wheel rims could definitely be sharper; the image generation pass left the rim textures a bit soft. But because it’s a sweeping drone shot, that slight softness actually works in our favor since it passes naturally as motion blur and lens depth.

Ultimately, it saved the shot in less than a day. Blending quick generative tools with basic compositing logic is becoming an incredibly powerful way to handle these kinds of last-minute continuity headaches.