Why do AI agents get video wrong, even when the code is right?
Article by Pedro Henrique Lahr
AI agents get video wrong because they work blind. They build the video page, render the file and get back an MP4 they cannot watch. The code compiles and runs; what goes wrong is the relation between two things on screen, like a title sitting on someone's face or a zoom that snaps back mid-sentence.
The code is right and the video is wrong. How?
Because correct code does not mean a correct video. A title placed 400 pixels from the top is valid code. If the presenter's face is right there, the video is wrong. The mistake is not on any line: it lives where the title meets the face, on one particular frame.
In the analysis that started the Kodecz project, we looked at 27 defects that showed up in Konecz's own videos before delivery. 21 were relations between two things. Only 6 were something else, and those 6 repeated a lesson that was already written in the agent's instructions.
What are the five reasons?
1. The agent cannot see the video. It reads text and writes text. Without a preview it can open, every visual decision is a guess.
2. The mistake sits between two things. Text and face, zoom and speech, cut and word, letter color and the color actually behind it. Each part, on its own, looks fine.
3. The preview lies. In the Kodecz error taxonomy, the class «Font that never reaches the render» has a familiar symptom: the preview finds the font installed on the machine, the render does not. The title comes out wider and wraps onto two lines only in the final file. The same goes for animation tied to the browser clock: in the preview it looked right, in the render it jumps or shows up already finished.
4. Checking is done by sampling. Looking at every frame of a 60-second video at 30 frames per second means 1,800 images. The agent looks at three or four, and the mistake is in the stretch it skipped. That became a class in the taxonomy too: «Checking by sampling lets the error through».
5. A written rule is not a seen rule. The 6 repeated defects prove it. The rule «no text over the face» was in the manual. The agent read it, agreed and broke it, because it had no way to know where the face was on each frame.
Which mistakes show up most?
The research gathered 4,826 public reports and 107 pages of rules and guides and arrived at 106 kinds of mistake. Of those, 94 come from what the agent writes and 12 from the tool it uses. A few examples, with the reason the agent misses them:
| Mistake | Why the agent misses it |
|---|---|
| Text over the face | it does not know where the face is on each frame |
| Zoom that returns mid-sentence | it picked a fixed duration without checking where the sentence ends |
| Caption on two lines | it counted words instead of measuring width in the real font |
| Low contrast | it chose the color in code, without measuring the real background |
| Caption behind the Reels buttons | it designed for the whole frame, not for the phone screen |
| Cut in the middle of a word | it used the rough transcript timing, with no margin |
None of them is a syntax error. All of them are failures to look.
So would a longer manual fix it?
No. That was the most useful finding of the project: a longer manual would not have prevented the 6 repeated defects, because the problem was never a missing rule. The agent needed eyes.
That is why Kodecz treats the agent as someone who needs instruments, not more text:
- A one-frame preview in one to two seconds, so the agent looks at what it wrote.
- Reading from inside the render: the engine knows where each line of text is, where the face is and which word is being spoken on every frame.
- Automatic checks on every frame, returning the frame, the element and the reason, plus a picture of the problem.
With that, the rule in the manual becomes a measurement. The agent does not have to remember not to put text on the face: the check tells it on which frame it did.
What does this change if you buy AI-made video?
It changes the question you ask your provider. Instead of «is the AI good?», ask «what do you check before you deliver, and how?». At Konecz, the answer is the Kodecz rule set running on every video. The portfolio held 108 videos on November 19, 2026, across profiles like Lahrs, Vigia Pix and Ratomix, all assembled this way.
Frequently asked questions
Does a stronger AI model make fewer video mistakes? It makes fewer code mistakes, but relation mistakes stay as long as it cannot see. In the paper's test, Claude Opus with only the documentation got 22 of 139 cases wrong.
Do automatic checks replace a person? No. They catch what can be measured (faces, lines, contrast, timing). Taste and style still need someone looking.
Does this apply to any agent? Yes. Claude Code, Codex, Cursor, Gemini CLI or your own agent: all of them work blind unless someone gives them a way to see.
More in the summary of the paper on AI agents making video and on the Kodecz page. If you would rather get finished, checked videos, meet Konecz.