Incorrect IndexEntry.StreamOffset value in IAB Track Files

[palemieux]

ST 377-1, 11.1.4 states:

Stream Offset
Provides the absolute byte offset to the start of the indexed Edit Unit within the Essence Container or the indexed
external Essence file, such that
[...]
2. For clip-wrapped Essence, each Index Entry value shall mark the start of an Edit Unit of stored Essence,
excluding the KL at the start of the Essence Container in each Partition.
Note 1: Including the bytes occupied by KL at the start of the Essence Container would eliminate the ability
to use a single Index Entry to index CBE Essence.

It looks like the AS02 IAB writer indexes IA Frame from the start of the K of the Clip instead of from the start of the V, i.e. the essence container position of the first edit unit is 24d instead of 0d, :

asdcplib/src/AS_02_IAB.cpp

Line 225 in 69e1a22

this->m_Writer->m_StreamOffset = RESERVED_KL_SIZE;

It is relatively safe and simple for a parser to detect and correct an Incorrect Stream Offset.

Recommendation:

• fix the issue in ASDCPLib
• warn implementers, through a note in ST 2067-2 and/or through the IMF UG, that some files might be incorrect

[jhursty]

asdcplib typically writes 4-byte Length value, but there are some use cases where this won't do and the correct size of the KL encoding must be determined and used. This is one of those cases. It is happening here because the clip wrapping is producing a huge packet that requires a longer encoding of Length than is typically required (in frame-wrapped d-cinema @ 250 Mb/s, all packets were small enough for 4-byte Length encoding.)

[palemieux]

@jhursty This is a separate issue from #151 and #156. In this particular case, index entries contain offsets from the K of the Clip element, instead of from the V of the Clip element

[palemieux]

Similarly, neither the index reader nor the IAB module add the size of the KL to the file offset:

asdcplib/src/AS_02_IAB.cpp

Line 522 in 9baa76f

ReadFrameImpl(ui32_t frame_number, ASDCP::FrameBuffer& frame, ReaderState_t& reader_state, AS_02::h__AS02Reader *reader, bool reallocate_if_needed) {

asdcplib/src/h__02_Reader.cpp

Line 347 in 9baa76f

AS_02::MXF::AS02IndexReader::Lookup(ui32_t frame_num, ASDCP::MXF::IndexTableSegment::IndexEntry& Entry) const

[palemieux]

FYI. I have seen IAB files in the wild that exhibit this bug, and at least one major tool expects stream offsets to be indexed for the K of the clip-wrapped essence container.