Miscellaneous issues

[videoh]

I try to use the windows Avisynth plugin. The following issues arise for me:

1. The open time is 30-40 seconds every time. More seriously, it hangs forever when opening a good proportion of the times I try.
2. I returned just the base view but frame ordering is wrong: one frame forward, another forward, one backward, forward, forward, backward... This is an absolute showstopper.
3. It would be good if you could accept separate base and mvc streams. That's what my demuxer gives me and then I have to combine them using a utility.

The script:

loadplugin("libavsmvc.dll")
MVCSource("combined.264",stack="base") # PROMETHEUS_3D

[jens-duttke]

Thanks for the report. Point by point:

1. Base-view frame ordering ("forward, forward, backward").
I was able to reproduce that and fixed it. It was a display-order regression - adjacent frames emitted in decode order instead of display order when the base view is coded out of display order. Please grab v0.4.2; I verified the shipped libavsmvc.dll now decodes the base view bit-exact to FFmpeg on Prometheus:
https://github.com/jens-duttke/mvc-source/releases/tag/v0.4.2

2. Open time / "hangs forever".
The 30-40s open is a one-time scan of the whole elementary stream to get an exact frame count. A raw .264 carries no index, so it has to be read once (same idea as ffms2/L-SMASH indexing), which scales with disk speed. The "hangs forever" I can't reproduce here (many opens on a Prometheus 3D sample, single- and multi-threaded, no hang). To narrow it down:

• does it hang during the initial wait, or after, while decoding?
• what drive is the file on (local SSD/HDD, or a network/NAS share), how many CPU cores, which OS?
• which build/release are you on?

3. Separate base + MVC inputs.
That's planned - it removes the manual combine step. If you can send me your separate base + dependent streams and the exact command/tool you use to combine them, that gives me the input for this feature and also lets me reproduce the combined.264 that's hanging (point 2).

[videoh]

Thank you Jens. 0.4.2 fixes the ordering issue. I understand the need for indexing but it should be saved on disk to be used for subsequent openings, as is done by the other major source filters. I haven't seen a hang yet with 0.4.2. I'll keep an eye on it.

I note that seeking is quite slow compared to other source filters. With a proper index it shouldn't have to decode more than one or two GOPs, depending on if it is an open-GOP stream.

Bravo for the great work you've done on the project!

[videoh]

BTW, I test seeking by putting Reverse() at the end of my script. That forces every frame to be a seek, or what I call random access. When I do that with your filter it never displays any frames. For my own source filter DGSource() it simply plays backwards like it should.

EDIT: I guess seeking to near the end is causing a hang. I'm not sure how close to the end you have to be but seeking to the last frame always fails.

[jens-duttke]

Two things from this thread are in v0.4.4:

On-disk index. The frame-count scan is now cached in a sidecar next to the source (<file>.mvcidx, keyed on size + mtime), so reopening skips it - a full read down to a few tens of ms. A stale or read-only case just re-scans, never a wrong frame.

Decoded-frame cache (~128 MB) that survives the decoder reset a seek needs, so backward access comes from RAM. Reverse() plays and seeking to the last frame returns it, both bit-exact.

Seek points are at IDRs for now, so short-GOP streams take a GOP or two, but some 3D Blu-rays run 600+ frames between IDRs and a cold jump there re-decodes from the IDR, so it's slow. Open-GOP-aware seek points are next.

[videoh]

The on-disk cache is working, thank you.

Seeks to near end of file still hangs. So Reverse is a no-go. Using 0.4.5.

[jens-duttke]

Ok, got an update on the seeking, and a correction to my "open-GOP-aware seek points are next" above.

I reproduced it, and it's not a hang - it's the re-decode cost. Seek points are at IDRs, and some 3D Blu-rays run 600+ frames between IDRs, so a cold jump near the end re-decodes from the previous IDR; the old fixed cache held only ~20 frames, so Reverse() re-seeked roughly every 20 frames.

I did try recovery-point (open-GOP) seek points, but they don't work cleanly on a raw .264: cold-decoding from a recovery point gives the right pictures for the trailing frames, but without the previous GOP in the DPB the reorder differs slightly right at the boundary, so a position-based index lands on the wrong frame there. Pinning frames down exactly would need the container timestamps a raw elementary stream doesn't carry, or a full decode-index (the slow-open route ffms2/L-SMASH take, which I wanted to avoid).

So instead, two changes make Reverse() usable without changing any output:

• cachesize (new option, MiB, default 512): the decoded-frame cache is now caller-sized. A larger cache spans a whole GOP, so a backward pass triggers one cold re-decode per cache-window instead of one per frame - raise it to span your longest GOP.
• Multi-threaded decode is now on by default, which makes a seek's forward re-decode much faster (still adjustable with the threads argument).

On a real 3D Blu-ray a Reverse() near the end went ~20x faster at the defaults, more with a bigger cachesize, and seeking still lands on the same frame you'd get playing forward.

It's in v0.4.6 which I've released a few minutes ago.

[videoh]

Thank you for the update. It's better but still performing poorly for me. With reverse() it takes 30 secs to show the first frame. And then when I hit play (in VirtualDub2) it runs fast until frame 85 and hangs for another 30 secs. Then again at frame 171, 257, etc.

I have attached an index file from DGIndexNV that shows the GOP structure. You can go to the end of the index and see that the last GOP is started by RPS/I and contains 19 frames. So a seek to the end should be lightning fast.

The technology for fast accurate seeking is well-developed in DGDecNV [DGSource()]. To get similar performance you MUST support RPS/I GOP starts. DGSource() does so and optionally it can treat a bare I (no RPS) as IDR because in most cases it is fine (starting one GOP back avoids issues -- I can tell more about that if needed). But RPS is non-negotiable. You have to support it, and that will solve all the issues. Another thing DGSource() does is handle SPS/PPS intelligently. I can give more details if you need them. Yes, DGSource() requires full indexing of the source stream and I think you should do the same.

Ultimately, it's your choice how well you want your seeking to work. A working filter for MVC, even if only supporting linear play, is already an amazing accomplishment. I can only point out that the technology is well-known and if you want to claim fast accurate seeking you can't just go halfway IMHO. If a bluray player had to stop for 30 seconds on a seek it would not be acceptable. I suggest that you support RPSs and do intelligent SPS/PPS management, and thereby get blazing fast seeks to any frame. I'm not sure why you are running into issues with RPS but DGSource() and others, as well as bluray players themselves, use them without issues. I'd be happy to share the DGSource() design with you if needed.

One last thing. A human readable index file would be greatly preferred by users. You can see how useful the attached index file is.

PROMETHE#00800 PID 4113.dgi.txt

[jens-duttke]

I've just released v0.5.0 which adds the recovery-point seeking you were asking for.

Seek points now land on every random-access point instead of only on IDRs, so an open-GOP recovery point (a non-IDR I picture behind a recovery_point SEI with recovery_frame_cnt 0) is a valid seek target now. That matters because IDRs are sparse on a 3D Blu-ray - on the Prometheus 3D sample I have (the same title your .dgi is from) there are 341 recovery points to 103 IDRs, with the IDRs running up to 787 frames apart and recovery points roughly every 20, so the worst-case re-decode a cold seek has to do drops from those 787 frames down to about 20. That's the same RPS/I spacing you pointed at in your .dgi.

The tricky part was landing on a recovery point at the right frame: its leading pictures come after it in decode order but before it in display order, so the index scan now reads the picture order count out of the slice headers (no actual decoding) to work out each recovery point's real display index. If a stream uses something the POC parse doesn't handle - field coding, an MMCO5 reset - it quietly falls back to IDR-only seeking, which is slower on a long GOP but never returns the wrong frame.

I also changed how parameter sets are handled on a seek, which you'd brought up: instead of re-feeding every SPS/PPS ahead of the target, it now feeds only the ones still active there. On the same sample that's 4 sets instead of 2173, and since they're spread across the whole file, on a hard disk that re-feed was actually the bulk of a cold seek's time.

Numbers on that Prometheus sample, my machine, at the defaults, against v0.4.6:

• cold seek to the last frame went from 4.25s to 0.18s
• Reverse() went from about 52 to 155 fps
• the worst single-frame stall, the part that reads as a hang, went from 2.21s to 0.19s

Two things you raised that I haven't done:

• A bare I with no recovery_point SEI still isn't used as a seek point - I only index IDRs and the SEI-marked recovery points, since the SEI is the one signal that guarantees a clean cold start. The one-GOP-back handling for a bare I is a fair next step; I stopped at the SEI because every disc I've tested marks its entry points with it, so I had nothing to test the bare-I path against. If you have material whose entry points are bare I with no SEI, that's exactly the case, and I'd want your input on it.

• The sidecar is also still a binary .mvcidx rather than a readable index like your .dgi. Fair point that the value there is being able to read the structure the indexer found; that could be a later addition.

[videoh]

Works great now! Reverse() is fine and plays fast. You shouldn't need bare I seek points. I haven't seen them on MVC streams in any case. I support it in DGSource() just for some rare ES files. It can be turned on/off.

Thank you for your great support!

[videoh]

I tried separate streams:

MVCSource("base.avc",dependent="dependent.mvc", stack="base")

Problems:

1. Takes much longer to open.
2. An index file is not created.
3. Seek to end is very slow, not instant as with combined streams.

[jens-duttke]

I've just released v0.6.0, which addresses the first two of your three points.

You're right about the cause - I originally built the two-file (separate base + dependent) support for BD3D2MK3D, which just decodes a stream straight through from start to end and never seeks, so an index wasn't needed there. For consistency I've now added it here too, the same on-disk .mvcidx sidecar the combined-stream path already had.

Why the first open takes a while: with two separate elementary streams there's no single decode order to read directly - the base and dependent NALs have to be walked and interleaved per access unit before anything can be decoded, which means a full pass over both files. v0.6.0 caches the result of that pass next to the dependent stream, so it only happens once; every open after that just loads the sidecar. On one of my larger test files (a full 3D Blu-ray, base + dependent together just under 40 GB) that first pass took a bit over three minutes; every open after that is under half a second.

Your third point - seeking to the end still being slow - isn't fixed by this. The two-file path currently only places seek points at IDRs, not at the recovery points the combined-stream path got in v0.5.0, so a cold seek there still re-decodes from the last IDR. Extending recovery-point seeking to two-file streams would need the picture order count derived across the two interleaved streams rather than one, which is more involved than the caching above - more than I have time for right now, so it'll have to wait a bit.

[videoh]

I confirm that the index is now created and working. Thank you! With that the long open time is less significant. I don't understand your point about RPSs and POCs.