Why this exists.
I had folders full of long videos and one recurring problem: finding "that scene". Every player gives you the same tool for it: a seek bar you drag back and forth while squinting at one tiny preview. For a 2-hour file that is a terrible interface. You are searching a 2D space (what the picture looks like) with a 1D control (a line).
VamSeek started as a single question: what if the whole video was laid out as a grid of thumbnails, so you find the scene by looking instead of dragging? Click the cell, you are there. That is still the core. Everything else (keyframe tricks, scene splitting, the clip tool) exists to make that grid fast and honest.
It is a Windows app on the Microsoft Store, with a free browser demo at /demo/ (30 minutes per file, nothing uploaded, every thumbnail is made on your machine).
This thread is where I will write down the ideas and the dead ends. Ask anything. Criticism is fine.
The 2D seek.
The grid is evenly spaced in time by default: every cell is N seconds (15 s is the sweet spot, 3 s is heavy). You can also split "by scene", where cells begin where the picture changes, so cells have different lengths.
A small red marker shows where playback is, and it moves across the grid as the video plays. Click a cell to jump; hold and drag to scrub finely across cells. Arrow keys walk cells, Home/End jump to the ends.
The subtle part is the mapping between time and position. A cell owns the time from its left edge to its right edge, the marker is a continuous function of time, and the thumbnail is the keyframe nearest the cell's middle. I tried a prettier-sounding rule once ("the middle of the cell is the start of the cell") and it broke the moment cells had different lengths: half of every cell became a place the marker could never stand, so it jumped. Lesson: the marker is a continuous map from one line of time onto a sequence of boxes. Any rule that leaves gaps in that map will jump somewhere.
Speed, part 1: thumbnails are decode-bound.
Making a thumbnail at an arbitrary second is not "read one frame". Video is stored as keyframes with long chains of difference frames (a GOP) after each. To show second 61.4 the decoder has to start at the previous keyframe and decode every frame up to the target. For an AV1 file with a long GOP that can be 150+ frames per thumbnail, and AV1 on a GPU without an AV1 decoder is pure CPU work.
I benchmarked three approaches against the naive one (seek a hidden <video>, wait, draw to canvas): WebCodecs, spawning ffmpeg per frame, and snapping to keyframes. All three came out within noise of 1.0x. The bottleneck was the number of frames decoded between keyframe and target, and that number does not depend on which API you call. The only thing that helped at the time was caching more decoded grids in memory.
Side finding that cost me a day: with WebCodecs you must close() every VideoFrame you do not keep. Hold a few and the decoder's frame pool runs dry and flush() hangs for 15 seconds.
Speed, part 2: the keyframe-only grid.
The breakthrough was changing the question. Instead of "show me exactly second 61.4", ask "show me any keyframe that falls inside this cell's time window". A keyframe decodes alone, no chain, so each cell costs one frame.
Measured on a laptop GPU, full grid of 150 cells:
- 1080p H.264, 15 min: 8.8 s -> 0.68 s (13x)
- a real 25-minute 10-bit HEVC file: 200 of 247 cells had a keyframe in their window and appeared in about one second
The 47 cells without a keyframe in their window fall back to the slow path (decode from the previous keyframe up to the window), spread across a few decoders, and fill in a couple of seconds later. The picture you see is "the nearest keyframe", which is usually within a second of the cell's time, and I show the real time under it so nothing lies.
To do this you need an index of where the keyframes are, which means reading the container's own tables (MP4's sample tables, MKV's cues) instead of asking the player. That index is also what makes the clip tool possible (next post).
Playing while the grid is still building.
First version: start playback while thumbnails were loading and the video stuttered, and the marker stuttered too. Two separate causes.
1) The thumbnail decoders and the main player were fighting over the same hardware decoder. Fix: the moment playback starts, the fast keyframe path yields. Remaining cells go through a single slow extractor until you pause. The grid finishes later; the video stays smooth.
2) The marker jitter came from somewhere else entirely: synchronous file reads in the main process. On an SSD a read is under a millisecond and you never notice. On a spinning disk each seek stalls the process for 100+ ms, which froze the timer that moves the marker. Making those reads asynchronous cut the worst marker interval from ~200 ms to ~70 ms.
Measurement trap worth sharing: if you benchmark the same file twice, the second run is served from the OS file cache and the HDD effect vanishes. Alternate between unread files of the same kind.
Clip mode: cutting without re-encoding, without ffmpeg.
You mark scenes to keep (or gray out parts to cut) on the grid, press save, and get a new MP4/MKV in a second or two. No quality loss, no bundled ffmpeg.
How: the compressed samples are copied as-is from the source into the new file, and only the index (MP4's moov, MKV's cues/clusters) is rebuilt for the kept range. The start of a cut snaps to the previous keyframe, and the UI snaps your handle there too, so what you see is what you get. Audio and video are aligned with an edit list so they start together even mid-sample.
Two things that bit me:
- Blu-ray rips with PCM audio have hundreds of millions of audio samples. Expanding the sample tables into arrays crashes; you have to keep them as run-length tables and binary-search.
- One file had its first video frame timestamped 2 frames after the edit list said playback begins. Spec says show black for 83 ms; ffmpeg (and therefore every Chromium player) instead shifts the first frame to zero. I match that, otherwise the saved clip starts 83 ms off from what you saw.
Containers: why AVI will not open (yet).
The player core is Chromium's, and Chromium only demuxes MP4/MOV, MKV/WebM and Ogg. It does not matter that your AVI contains plain H.264 inside; AVI, ASF, TS and FLV fail at the container level before any decoder is asked.
I built a fix: remux on open (copy the streams into a temporary MP4, no re-encode) with a small LGPL ffmpeg. It worked and took 0.1 s for an 8-second file. Then I tried my own AVIs: 4 to 10 GB each. A copy is proportional to size, so that is 20-60 s on an SSD and minutes on an HDD, every time, before you see a frame. Not acceptable, so I reverted it the same day.
The real fix is to stream instead of copy: remux into fragmented MP4 on the fly and feed it to MediaSource, so playback starts in about a second and seeking restarts the remux from the new position. That is designed, not built. For now AVI is "a container I cannot open" and MPC-HC does that job.
Smaller things, and dead ends.
- Columns are modes, not just density. 1-2 columns is for watching, 3-4 is the balance, 5-6 turns the layout into a search tool (folder tree, video cards, grid, player tucked in a corner). The menu groups them that way.
- Scene splitting ("By Scene -> Auto") picks the granularity per video: short or calm videos get fine cells, long busy ones coarse. Runs of similar frames merge into one cell.
- A mini player lets you keep watching one video while browsing another's grid. The trick is to not recreate the <video> element, so playback never hiccups while the panel floats.
- Dead end I liked: making the marker roll like a ball while you drag, so you can feel how much time a wide cell represents. It was cute during drags and annoying during playback. Removed, but the idea (show the time between cells with motion) is kept.
- Privacy: nothing leaves your machine. The browser demo reads files through the File System Access API and never uploads.
That is the state of things. Replies welcome. If you are a developer, I am happy to go deeper on any of this.