Claude Eyes: Search Video Moments and Measure an Edit

A transcript can tell you what a presenter said, but it can miss the button they clicked, the command typed on screen, or the instant a layout changed. In Moritz Kremb's Eyes demo, he gives a coding agent a repeatable way to inspect those details: index a video once, search its speech and on-screen text, then request a closer look where needed. The result is useful evidence for a video question, not a new video-native model inside Claude.

Watch Moritz's Demo

Creator: Moritz Kremb. This guide follows his demonstration and checks commands and limits against the project README. The video is a product demo, not an independent accuracy or cost benchmark.

What Eyes Actually Sees

A coding agent does not gain continuous vision simply because you paste it a YouTube link. Eyes converts video into artifacts the agent can read. According to its documentation, eyes index stores a local cache at ~/.cache/eyes/<id>/, so a follow-up question can reuse the same index instead of starting over. The tracks cover:

• Pictures and cuts: sampled, timestamped frames arranged into contact sheets, plus detected shot changes.
• Words said: available captions or a local speech transcript with word-level timing.
• Words shown: OCR text with its screen position and time, including labels the narrator never mentions.
• Sound and color: loudness, silence, beat estimates and palette measurements. These are signals, not full musical or emotional understanding.

The viewer, eyes view REF --open, puts these tracks over a player for the human reviewer. Sampling is adaptive, not exhaustive: the README lists roughly 0.5 to 4 frames per second for the main index. For fast motion or small text, the agent needs a burst, exact frame request or high-resolution crop before making a confident claim.

Four Jobs the Demo Makes Concrete

Find something only shown on screen

At 06:02, Moritz asks for the point in an earlier Grok Bot video where a Pages feature appears. The answer gives a time window around 03:54 to 05:00 in that other video, even though his spoken description did not name the feature. This is the key advantage over a transcript-only search. Verify the returned moment; OCR can misread tiny or stylized text.

Map layout changes

At 08:06, Eyes describes three screen arrangements and reports eight changes in his example. That is useful for a creator studying presentation rhythm: when does face-to-camera give way to a full-screen demonstration? A scene detector can still mistake a major graphic change for a cut, so treat the count as a starting measurement.

Study a quick animation

At 09:25, he uses burst to collect eight frames across one second of a transition. The denser sequence reveals steps a normal contact sheet might skip. It helps describe timing and motion, but it is not the same as seeing every frame or inheriting permission to copy another creator's artwork.

Compare editing “DNA”

At 10:35, a dna report turns a reference edit into numbers such as cuts per minute, median shot length, caption density, loudness and palette. eyes compare can then mark a new render as match, close or off against those metrics. That is a practical quality check for an editor; a matching score does not tell you whether the story, clarity or accessibility works for the audience.

Install and Try One Public Video

The README supports macOS, Linux and Windows with Python 3.10 or newer. Read the repository's setup instructions before letting an agent install system tools. For a command-line setup with uv available:

uv tool install git+https://github.com/moritzkremb/claude-eyes
eyes setup --fix
eyes install-skill
eyes setup

eyes setup --fix may install FFmpeg, whisper.cpp and a speech model where it can. Restart the agent after installing its skill. If you prefer an agent-led setup, the README suggests asking it to install Eyes from the repository; review each proposed command and its permissions first. Installing only the skill with npx skills add moritzkremb/claude-eyes is another documented route, but the CLI is still needed when the skill first runs.

Start with a public or authorized video. The skill can invoke Eyes from a prompt such as /eyes followed by a URL. The underlying CLI also works directly:

eyes index "https://www.youtube.com/watch?v=mOy7JQjTYwE"
eyes search "layout" --in mOy7JQjTYwE
eyes view mOy7JQjTYwE --open

For a narrow visual question, try eyes burst REF 0:10.5 or eyes crop REF 12:02.4 --box 0,0,0.5,0.5. Replace REF with an indexed video's unique ID or title fragment. The command reference also documents transcript, frames, library, report and JSON output.

What “100% Free” Does and Does Not Cover

Eyes itself has an MIT license and runs locally by default. Moritz's 13:27 cost slide estimates model-reading charges of about $0.01 to $0.10 for a 45-second clip, $0.20 for two minutes and $0.30 for twenty minutes with his top-model assumptions. Those are illustrative estimates, not a published all-in bill for your machine, agent plan, transcription needs or repeated queries. The repository does not publish a general cost guarantee.

The privacy default is local video and audio processing. If you explicitly select --transcriber elevenlabs, Eyes uploads the audio track for transcription and that service can charge separately. A coding agent's own hosting and data policies also matter. Only index material you own or are authorized to process; a public video is a simple first test, while client footage deserves a permissions check.

Where to Slow Down and Verify

Eyes makes video inspection repeatable; it does not make it infallible. The README says slow dissolves can be missed, split-screen graphics may be counted as cuts, and sampled contact sheets can omit brief details. Speech is handled as text by default, while music is represented mostly by measurements. Ask the agent for the exact source timestamp, open the viewer or original clip, and use a crop or burst before copying a command from a tutorial or judging a fast transition. When the question is purely spoken content, a transcript may be enough; Eyes earns its setup cost when the on-screen details or editing pattern matter.

The Watch skill guide describes a different video-analysis route, including a Gemini engine. Do not assume the two projects share installation steps, privacy settings or cost structure simply because both help agents work with video.

Video Chapters

Sources

Primary: Moritz Kremb's video and the supplied transcript; Claude Eyes README and code, its agent setup guide, roadmap and license. Checked 23 October 2026. The examples are creator demonstrations, not independently replicated tests.