# AI agents making video: where they fail and how Kodecz checks it

> When an AI agent writes a video as code, the mistake is almost never in the code: it is text over a face, or a zoom out of step with the speech. The Kodecz paper shows how the editor lets the agent see and check its own video, and measures it on 5,007 cases. A plain summary, with the numbers.

Pedro Henrique Lahr, Konecz, 2026-11-01. https://konecz.com/en/blog/kodecz-agentes

Kodecz is a video editor made for AI agents. This paper answers two questions: where agents go wrong when they make video, and what an editor has to give them so they can see and fix their own mistakes. The short answer: the agent almost never gets the code wrong. It gets wrong the things that only show up when two things meet on screen, like a title landing on the speaker's face.

## What is an AI agent?

An AI agent is a program with artificial intelligence that does tasks on its own: it reads files, writes code, runs commands and checks the result. Agents already make videos by writing a web page, which is then turned into an MP4 file, frame by frame.

The catch is that the agent does not watch the video. It writes text, asks for the video to be made and gets back a file it cannot see. It is like painting blindfolded and only hearing at the end whether it came out right.

## Where do agents go wrong?

We looked at 27 defects that turned up in our own videos before they were delivered:

- **21 were relations between two things:** text over a face, motion out of step with speech, a cut in the middle of a word. None of this shows up by reading the code alone.
- **6 repeated a lesson that was already written** in the agent's instructions. The rule was there. The agent just had no way to see that its video broke it.

So a longer manual was not the answer. The agent needed eyes.

Then we gathered public reports: 4,826 reports and 107 pages of published rules and style guides. The 27 defects grew into **106 error types**. Fifteen of them are the "looks AI-made" kind, like the same purple-to-blue gradient as always, or everything centered.

## How does Kodecz help the agent see?

The project's rule is simple: anything a person would see on an editing timeline, the agent can ask for as data or as a picture. In practice:

1. **Every frame has a fixed address.** Frame 120 is always the same frame, wherever the render starts. The agent looks, changes something and looks at the same frame again.
2. **Fast preview.** The agent asks for one frame and gets the picture in under 2 seconds (1.7 to 1.9 s), with the same pixels as the final video.
3. **Reading from inside.** Because the render engine is ours, it knows where every line of text is, where the face is and which word is being spoken in each frame.
4. **Automatic check.** One command checks the video and answers with the frame, the element and the reason, plus a picture of the problem. It was tuned on 17 of our own videos, approved and rejected: after tuning, no approved video fails, and every mistake in the rejected ones is still caught.

## How do we know it works?

We built the **Kodecz Bench**, a set of 5,007 tests. Each test is a correct scene with one mistake planted on purpose, like hiding an object in a picture for someone to find. The score comes from a fixed rule that reads the video from inside, not from another AI giving its opinion.

Each rule was also tested against the other mistakes: it flagged the wrong mistake in 99 of 11,025 crossings, 0.9%.

In the first run, one agent (Claude Opus) got only the documentation and got **22 of 139 cases** wrong, 15.8%. It cost US$ 0.53 per case. Editing was the area with the most mistakes: 36%. A newer run, with 300 cases, gave a similar number: 35 mistakes in 219, 16.0%.

The paper also proposes a five-layer way of talking to agents about video. It has not been measured yet, and the paper makes no claim about its effect.

## Frequently asked questions

**Is Kodecz only for Claude?**
No. Any agent that can write files and run commands can use it. The paper's test used Claude Opus, with a small pilot of Claude Sonnet.

**Does one AI grade another in the Kodecz Bench?**
No. The score comes from fixed rules that read the video from inside. The same test always gives the same answer.

**Does the check see everything?**
No. It reads the text of the page. Text drawn inside an image or a filmed video is not seen.

## Full paper

This post is a summary. The full paper, with the tables, the limits and the list of mistakes, is at [doi.org/10.6084/m9.figshare.34880928](https://doi.org/10.6084/m9.figshare.34880928). The code, the agent guide and the Kodecz Bench are at [github.com/konecz/kodecz](https://github.com/konecz/kodecz).

How to cite: Lahr, Pedro Henrique. *Kodecz: a video editor built for AI agents, how agents fail when they write video as code, and how to check it from inside the render*. Figshare, 1 Nov 2026. https://doi.org/10.6084/m9.figshare.34880928