Kling – Tan Zi Jian

July 8, 2026

AI vs. Physics: Can Video-to-Video Models Handle Fluid Refraction and Caustics?

As a visual effects artist, procedural fluid simulations are a staple in my workflow. Building custom velocity fields, tweaking flip solvers in Houdini, and dialing in physically accurate rendering for water takes time, computational power, and a lot of patience. With the rapid explosion of AI video-to-video models, a natural question arises: can these tools bypass the heavy lifting of traditional fluid sims?

Specifically, I wanted to test whether AI can accurately interpret water movement, the refraction of light through a volume, and water caustics. To find out, I set up a stress test comparing a traditional procedural workflow against two AI models: Seedance 2.0 and Kling 3.0.

Here is a breakdown of the experiment and the results.

The Control: Procedural Accuracy in Houdini

To establish a baseline, I built a controlled simulation in Houdini. The setup was straightforward but designed to test specific optical properties:

A small ocean geometry generating surface waves, placed over a simple grid.
A yellow cube placed on the grid, fully submerged under the water.
Materials accurately assigned to calculate the Index of Refraction (IOR).
Caustics explicitly turned on and lighting set to highlight the light patterns on the floor.

The scene was rendered in Karma. The very first frame of this render served as the ground truth (and also reference image) for how light should bend around the submerged cube and cast caustics on the ocean floor. The whole 3-seconds shot was also rendered out and composed for comparison with other Gen AI outputs.

The final shot that was made using Houdini and composed in After Effects.

The Methodology

To test the AI, I provided the models with the exact same starting point and motion data.

The Input Frame: The extracted first frame from the Houdini render.
The Reference Motion: An extracted flipbook animation (mp4) of the waves and the motion of the small ocean geometry.
The Prompt: Use ff as first frame. ff shows a yellow cube in shallow water with water caustics, distorted by refraction of light. Use Video1 as reference video for movement of water and position of cube. Interpret the distortion of the cube and water caustics. No audio.

The Flipbook render from Houdini used as reference video

This exact combination was fed into both Seedance 2.0 and Kling 3.0.

The Results: How Did the AI Perform?

Evaluating the outputs required looking past the initial “wow” factor of AI generation and critically analyzing the physics of the scene.

Seedance 2.0: Good Motion, Broken Physics

Seedance 2.0 output

Water Movement: Seedance did a surprisingly good job imitating the motion of the waves from the flipbook reference. It definitely looked like a fluid, and the pure white reflections scattered across the surface were convincing.
Material Properties: This is where the physics fell apart. Instead of rendering clear water in a pool, Seedance generated what looked like an opaque, sky-blue liquid.
Refraction: Because the model hallucinated a highly opaque liquid, the physics of light transport were completely illogical. A thick, opaque liquid shouldn’t allow light rays from a submerged cube to refract that clearly. Setting that aside, the distortion on the cube was extremely minimal compared to the Karma render. It appeared the model simply morphed the object toward the yellow square I left in the reference video for general positioning. (Note for future tests: removing the tracking square might give the AI more freedom to interpret genuine optical distortion rather than acting as a rigid morph target).
Water Caustics: Due to the opaque nature of the generated fluid, the caustics were almost entirely non-existent.

Kling 3.0: Total Hallucination

Kling 3.0 output

Output: The result from Kling 3.0 was completely unusable for this specific use case. The model entirely misinterpreted the prompt and reference, generating what looked like a frozen layer of ice with water running underneath it. While there were some vague caustic-like patterns on the pool floor and distortion of the cube, the overall output was a failed interpretation of the input data.

Conclusion

While video-to-video AI models are making incredible strides in tracking 2D surface displacement, this test highlights a massive gap in their ability to understand volumetric data and complex physical light transport.

The AI tools treated the reference as a 2D warping task. Seedance 2.0 successfully mimicked the surface fluidity but failed to comprehend the depth of the water, fundamentally breaking the material properties by turning a clear, refractive medium into an opaque liquid. Kling 3.0 simply hallucinated entirely different physical states.

Feature Tested	Seedance 2.0	Kling 3.0
Water Movement	Convincing surface tracking and fluid motion.	Unusable (hallucinated an ice layer).
Material Accuracy	Failed (rendered an opaque liquid).	Failed.
Refraction	Minimal; hindered by opacity and reference morphing.	Some distortion present but unnatural-looking.
Caustics	Barely visible due to incorrect material density.	Failed.

For now, if a project requires precise optical fidelity, where refraction and caustics need to accurately interact with submerged objects, it still absolutely demands a robust procedural solver and a physically based renderer. AI can approximate the broad strokes of movement, but true fluid physics remains securely in the realm of traditional 3D pipelines.

Generative AI Kling

Zi Jian

June 29, 2026

AI in the VFX Pipeline: Faking a High-Speed Police Chase

Sometimes, the biggest obstacle in visual effects isn’t the technical execution, it’s the logistics.

Imagine getting a brief that calls for a high-speed pursuit on a quiet stretch of highway, complete with a Korean police cruiser hot on the tail of a getaway car. The traditional route? Applying for permits to shut down a local road, finding a prop Korean police vehicle to rent, hiring stunt drivers, and navigating a mountain of red tape just to make sure you don’t break any major traffic laws.

Instead of dealing with that logistical nightmare, I decided to build the entire sequence in post using a hybrid workflow of traditional After Effects pre-vis and AI video generation. Here is a breakdown of my process.

Step 1: The Raw Plate and Asset Generation

I started with a simple, smooth tracking shot of an empty road.

The raw plate to be worked on

Next, I needed my vehicles. I sourced reference photos online for a standard black sedan and a Korean police car. To get these cars sitting correctly in the environment before generating the video, I used nano banana pro. I took one frame of the raw road footage and fed it into the AI along with the car photos, using this specific prompt:

“Place the police car on the road, travelling from left to right of the frame. Make sure the size proportion of the car to the road is correct and logical. Make sure the car plate and text are clear.”

I repeated this process for both cars until I had two clean, properly proportioned still images of the vehicles sitting exactly where I wanted them on the road.

The resulting reference images after going through Nano Banana Pro

Step 2: Old School Blocking in After Effects

AI video generators are powerful, but they often lack spatial awareness and timing unless you guide them. To fix this, I jumped into After Effects.

I loaded up the raw video footage and used the two AI-generated still images as a size reference. I then created simple 3D geometry in AE: a black cube to represent the getaway car, and a blue cube for the police cruiser. I animated these cubes flying down the empty road, matching the exact speed, scale, and trajectory I wanted for the final shot.

This gave me a flawless reference video to feed back into the AI.

The reference video with animated colored cubes

Step 3: Bringing it to Life with Kling 3.0 Omni

With my AE reference video and the two car images ready, I moved over to Kling 3.0 Omni. The goal was to have Kling look at the animated cubes and replace them with the photorealistic cars, while adding all the environmental effects.

Here was the prompt I used:

“Use Video as a reference. Replace the black cube with the black car in Image 1 and replace the blue cube with the police car in Image 2. Police lights flashing as the police car drives by. Heavy motion blur.”

After tweaking and refining the prompt twice, the output was incredibly solid. The AI nailed the heavy motion blur, the flashing police lights, and the raw speed of the pursuit.

The final output

The Catch: The Missing Seed Number

While the final result looked great, this workflow highlighted a significant limitation with current web-based AI tools.

I wanted to generate that exact same successful result again at a higher resolution. However, because the web UI for Kling doesn’t allow you to lock in or input a specific seed number, I couldn’t reproduce the exact same generation. I was entirely at the mercy of the random noise generation.

An example of a failed output

It is a great lesson in the current state of AI video tools: they can save you from a logistical nightmare and produce amazing results, but the lack of granular control, like seed retention, means you have to be ready to adapt when upscaling or revising shots.

Generative AI Kling

Zi Jian

June 26, 2026

From Greenscreen to a Realistic Urban Environment: A Hybrid AE + AI Workflow

Finding the sweet spot between traditional compositing and generative AI is currently one of the most exciting challenges in VFX. I recently wrapped up a shot that perfectly encapsulates this hybrid approach, relying on traditional spatial blocking in After Effects and letting AI handle the heavy lifting for the environment generation.

Here is a breakdown of the workflow, the tools used, and a few quirks I noticed along the way.

The Brief & Scenario

The Setup: A client needed a gritty, grounded cinematic shot for a tactical shooter promo.

The Action: A sniper positioned on a concrete rooftop takes a high-recoil shot at an off-screen target, set against an ordinary, quiet cityscape at night.

The Catch: The only supplied material was a single, static shot of an actor performing the action in front of a studio greenscreen. No environment plates or 3D environment assets were provided.

Green screen footage

Instead of building a matte painting or full 3D environment from scratch, I used this as an opportunity to test an AI-assisted pipeline using Nano Banana and Kling AI.

Step 1: The Foundation (After Effects)

Everything started in After Effects. To get a clean slate, I applied Keylight to pull the greenscreen, isolating the actor completely.

Keylight and garbage mask applied to footage

Since AI video generators need robust spatial context to understand depth and geometry, I couldn’t just feed the alpha channel into a generator. I built a rough 3D spatial block-out directly in AE to serve as a guide:

Grey 3D Cubes: Placed around the actor to map out the concrete rooftop and ledge.
Red 3D Cubes: Placed in the background to indicate the scale and placement of the distant apartment buildings.
Blue Solid: Placed at the very back to act as the night sky placeholder.

Step 2: Look Dev (Nano Banana)

With the spatial blocking complete, I exported the First Frame (FF) of this sequence.

I brought this FF into Nano Banana to establish the art direction. I prompted the model to interpret my colored cubes, turning the grey blocks into weathered, stained concrete, the red blocks into realistic brick apartment buildings with fire escapes and water towers, and the blue solid into an overcast night sky.

It took a few iterative generations, but the geometry blocking held up perfectly and guided the generation exactly where I needed it.

Step 3: Motion Generation (Kling AI)

This is where the process becomes incredibly interesting. I took the original reference video (the keyed actor interacting with the primitive colored 3D cubes) along with the finalized First Frame from Nano Banana, and ran them through Kling AI.

A big reason I used AI for this shot was to handle the transient effects. I prompted Kling to produce the muzzle flare, muzzle smoke, and dust particles exactly when the actor acted out the shot being fired. The environmental consistency was fantastic. The AI tracked the structural intent of the cubes and populated the realistic urban background beautifully behind the actor’s movements while successfully generating the gunfire effects.

Result from Kling generation

The Caveat: One interesting quirk I noticed was how Kling interpreted the actor’s final pose. The recoil itself matched the original footage perfectly, but at the very end of the action, Kling repositioned the rifle back to its original starting position. In the original greenscreen footage, the actor actually kept the rifle held slightly backward due to the recoil weight. It is a great reminder that while AI is excellent at style transfer and environment generation, strict kinetic accuracy still requires a watchful eye.

Step 4: Final Compositing (After Effects)

With the muzzle flash and smoke successfully generated by Kling, I brought the output back into After Effects for the finishing touches.

I tied the whole shot together with some optical glow, a subtle vignette, and a cinematic color grade to unify the AI-generated city with the actor’s original lighting.

Final output

Final Thoughts: Using primitive 3D shapes to control AI generation is an incredibly effective workflow. By defining the volume and depth explicitly in After Effects, you drastically reduce the AI’s tendency to hallucinate structural details, allowing you to focus entirely on art direction and finalizing the composite.

Consolidated workflow visualization

ComfyUI Generative AI Kling

Zi Jian

June 23, 2026

VFX Breakdown: Changing Youngji’s Red Hair to Black Using AI

Do you guys remember the Youngji red hair scandal during the local elections? She dyed her hair red, people started taking it as a political signal, and she had to rush to the salon, dye it back to black, and drop an apology. But what happens to the commercial spots she already shot? She had this KFC commercial running where she was still sporting the red hair. I thought this was the perfect scenario for some practice. I wanted to see if I could use AI to retroactively change her hair to black in a 3-second shot from that commercial. It’s kinda like a narrative behind the reason I chose this specific shot to practice on, treating it like a real-world post-production rescue mission to fix a finished asset without needing a reshoot.

The 3-second shot to be worked on.

Getting the First Frame

I started by taking the shot and extracting the very first frame using AE. At first, I used Nano Banana Pro to change the hair color, but several prompts later, her face kept changing. I decided to switch over to ChatGPT Images 2.0, and it was one prompt, one kill. It gave me the perfect starting frame with black hair while perfectly preserving her actual face.

Nano Banana Pro kept giving me Rose from Blackpink and couldn’t keep the rest of the scene consistent.

ChatGPT Image 2.0 was able to produce the First Frame that I needed.

Generating the Video in Kling

With the first frame ready, I went to Kling and used the First Frame plus Reference Video method. My prompt was to change the subject’s hair color in the video to black, using my newly generated image as the first frame. I hit my first obstacle here because the reference video was too short for Kling. It was about 60 frames for a 24 fps video, but Kling needs a minimum of 72. So, I jumped back into AE and freeze-framed the front and back for a few frames to hit that 72-frame requirement. I put the extended clip back into Kling and ran it. The generation was one try, one kill.

Before and After Kling 3.0

Masking in ComfyUI

Even with a great AI generation, I didn’t want to just paste the new video over the commercial and ruin the background and all the KFC branding. I needed to isolate her. I booted up ComfyUI and used the RMBG node on the BEN2 model to extract just the mask of Youngji from the original video. Well, I could have rotoscoped her out in AE directly, but where is the fun in doing it manually when I can throw it to AI to do it quickly? Furthermore, all I needed was a rough mask, so RMBG within ComfyUI was more than enough.

Used RMBG (BEN2) to get a mask image video.

Final Comp in AE

For the final step, I brought everything back into AE. I used a luma mask to remove the dark parts of the mask and added a feather to soften it. Then, I used minimax to increase the area of the non-alpha channel so the new mask would completely cover her original red hair. Finally, I overlayed the Kling footage of black-hair Youngji over the original video, using a Track Matte of the edited mask of Youngji to get the final output.

The whole process in a nutshell

The final result seamlessly replaces the controversial red hair while keeping the original environment and lighting completely intact. It really goes to show how integrating generative AI tools into a standard workflow gives you a ton of creative control and efficiency, especially if you ever need to save a campaign from a sudden PR issue.

Generative AI Kling Seedance

Zi Jian

June 18, 2026

The Last-Minute PPL Pivot: Swapping a Car in a Drone Shot with AI & Compositing

So, here’s a classic indie film scenario. You finally lock picture on a scene, everything is tracking nicely, and the director is happy. Then the producer walks into the post suite with a massive grin because they just closed a last-minute Product Placement (PPL) deal with BYD. The catch? The car we actually shot on location during a smooth, 5-second aerial drone sequence is a bright blue SUV. Now, it absolutely has to be a white BYD car.

The 5-second aerial drone sequence to be worked on.

Instead of booking an expensive reshoot or spending days on a full 3D asset tracking pipeline, I wanted to see if I could solve this using a generative AI workflow while keeping the original Full HD plate completely crisp.

Here’s how the process went, the roadblocks I hit, and how I blended AI with traditional compositing to make the shot work.

Avoiding the “AI Hallucination” Trap

When you’re doing a specific vehicle swap for a brand, your biggest hurdle is fidelity. If the AI shifts the contours or gives it a weird generic shape, the illusion falls apart instantly. Since newer EV models like BYD aren’t heavily represented in standard base AI models, my biggest concern was making sure the model wouldn’t hallucinate a totally different car.

I figured the easiest path would be a First Frame + Video-to-Video workflow.

First Frame + Seedance 2.0

I pulled the first frame of the drone shot, grabbed a reference photo of the white BYD car from the internet, and used Nano Banana Pro to generate a clean, structurally accurate first frame.

Then I tossed that generation into Seedance 2.0 for the video-to-video pass. The car reproduction was solid, and it actually nailed the layout of the wheels. However, the environment continuity completely broke down. After a few frames, the background would suddenly jump out of alignment, making the footage completely unusable for a continuous camera move.

Output produced by Seedance 2.0.

Switching to Kling

I ran the exact same first-frame setup through Kling instead. Kling handled the camera tracking perfectly, and the spatial continuity matched the smooth, sweeping drone move exactly.

The downside was the overall image quality of the output. The original footage had a lot of motion blur from the drone’s speed, and the model struggled to upscale those fine details cleanly, leaving the plate looking pretty muddy.

Output produced by Kling.

Cutting Out the AI Noise

At this point, you don’t just throw your hands up; you use traditional VFX logic. If the AI environment looks messy but the car tracking is spot-on, you isolate the car and throw away the rest of the AI noise.

To keep the original Full HD drone plate completely clean, I built a quick compositing bridge. I rotoscoped just the white BYD car out of the muddy Kling render and added a subtle feather to the mask edges so it would transition naturally into the asphalt.

Then, I dropped the isolated AI vehicle directly on top of the original blue SUV on the untouched Full HD plate. To lock it into the scene, I matched the black levels, highlights, and color temperature of the white car to the ambient lighting of the original plate.

Output after compositing Kling 3.0’s car output over the original footage

Final Thoughts

The final composite looks completely convincing in motion. If I’m nitpicking, the wheel rims could definitely be sharper; the image generation pass left the rim textures a bit soft. But because it’s a sweeping drone shot, that slight softness actually works in our favor since it passes naturally as motion blur and lens depth.

Ultimately, it saved the shot in less than a day. Blending quick generative tools with basic compositing logic is becoming an incredibly powerful way to handle these kinds of last-minute continuity headaches.