Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 25 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,31 @@ the public-API contract.

## [Unreleased]

_Nothing yet._
### Fixed

- **A transcoding audio bridge that produces nothing now says so, instead of dying as a muxer error two
subsystems downstream (AE#396).** A plain SD MKV with mono MP3 audio failed on the native route at
nine of nine start positions, ending on `Source audio cannot be muxed (code -22)` after three
identical revive attempts. The muxer was right and innocent: FFmpeg's mp4 muxer can only build an
AC-3/E-AC-3 sample entry from a packet that has been written, and for a bridged source those packets
come from the bridge's encoder, which had emitted none. Nothing anywhere in the session said that.
Every step between a source packet and an encoded frame ends in a `return` or in a loop that stops on
a negative code (a packet the decoder rejects, a decoder that answers nothing, a resample that
converts to zero samples, an encoder that keeps its output), which is correct per packet and silent in
aggregate, so a bridge that emitted nothing for a whole first segment was indistinguishable from one
that had simply not been asked yet. `AudioBridge` now counts each of those arms and keeps the
decoder's own error code, reports once (`AE#396 the bridge has produced no encoded audio at all`) as
soon as enough source has gone in for the silence to be structural, and the deferred first cut prints
the bridge's account instead of announcing a prime scan it does not run on this path.
- **A bridged session whose audio decoded to nothing fails immediately and truthfully, rather than
spending its revive budget re-reading the same bytes.** A producer restart rebuilds the muxer and
re-opens the encoder, both downstream of a failing decoder, so the same bytes were read three times
for the same answer. Zero decoded frames now ends the session at once; frames decoded with no packets
emitted is the encoder side, which a rebuild does heal, and keeps its revive.
- **New `PlaybackErrorKind.audioBridgeProducedNoOutput` for that failure.** It used to arrive as
`.vodSourceFailed`, which reads as "the source is gone" and ends a host's fallback ladder; the source
is neither gone nor unreadable here, and a second player that decodes the track itself plays the file.
Hosts with a ladder should demote on this kind, not stop.

## [6.30.2] - 2026-08-18

Expand Down
139 changes: 133 additions & 6 deletions Sources/AetherEngine/Audio/AudioBridge.swift
Original file line number Diff line number Diff line change
Expand Up @@ -413,6 +413,81 @@ final class AudioBridge: @unchecked Sendable {
/// on the pump thread; read lock-free for diagnostics, mirroring `liveBytes`.
private(set) var outputBytesLifetime: Int64 = 0

/// AE#396: what the bridge has actually done with what it was handed.
///
/// Every step between a source packet and an encoded frame can fail per packet, and each of those
/// arms ends in a `return` or in a loop that stops on a negative code: a packet the decoder
/// rejects, a decoder that answers nothing, a resample that converts to zero samples, an encoder
/// that keeps its output. Per packet that is the right behaviour, one bad frame must not end a
/// session. In aggregate it is the worst failure this class has, because a bridge that emits
/// NOTHING for a whole first segment surfaces as movenc's -22: the mp4 muxer is asked for an
/// AC-3/E-AC-3 sample entry it can only build from a packet that was never written, the cut
/// fails, the revive re-reads the same bytes twice more, and the session ends on "Source audio
/// cannot be muxed". Neither that sentence nor any line before it mentions audio decoding, so the
/// only subsystem that is definitely innocent is the one everybody reads about.
///
/// These counters exist to name the arm. Written under `opLock` on the pump thread, read
/// lock-free for diagnostics, exactly like `outputBytesLifetime`.
struct FeedStats: Sendable, Equatable {
var packetsFed = 0
/// Source packets `avcodec_send_packet` rejected and `feed` skipped (`invalidData`, `einval`).
var packetsRejected = 0
var framesDecoded = 0
/// `avcodec_receive_frame` answers that were neither a frame nor EAGAIN/EOF.
var decodeErrors = 0
var lastDecodeErrorCode: Int32 = 0
/// Decoded frames that never reached the FIFO: no samples, a null plane, or a resample that
/// produced nothing.
var framesDroppedBeforeFIFO = 0
var samplesEnqueued: Int64 = 0
var packetsEmitted = 0
/// `avcodec_receive_packet` answers that were neither a packet nor EAGAIN/EOF.
var encodeErrors = 0
var lastEncodeErrorCode: Int32 = 0

/// Fed real source, answered with nothing at all.
var isSilent: Bool { packetsFed > 0 && packetsEmitted == 0 }

/// The DECODER is the arm that failed, which no rebuild can change: a producer restart opens a
/// fresh encoder (#99 failure mode B) but hands the same decoder the same bytes. That is the
/// line between a muxer revive that can heal and three attempts that read identically.
var decodedNothing: Bool { packetsFed > 0 && framesDecoded == 0 }

var summary: String {
var parts = ["fed=\(packetsFed)", "decoded=\(framesDecoded)",
"enqueued=\(samplesEnqueued)", "emitted=\(packetsEmitted)"]
if packetsRejected > 0 { parts.append("rejected=\(packetsRejected)") }
if framesDroppedBeforeFIFO > 0 {
parts.append("droppedBeforeFIFO=\(framesDroppedBeforeFIFO)")
}
if decodeErrors > 0 {
parts.append("decodeErrors=\(decodeErrors)")
parts.append("lastDecodeError=\(FFmpegErr.text(for: lastDecodeErrorCode))")
} else if packetsRejected > 0, lastDecodeErrorCode != 0 {
parts.append("lastDecodeError=\(FFmpegErr.text(for: lastDecodeErrorCode))")
}
if encodeErrors > 0 {
parts.append("encodeErrors=\(encodeErrors)")
parts.append("lastEncodeError=\(FFmpegErr.text(for: lastEncodeErrorCode))")
}
return parts.joined(separator: " ")
}
}

private var stats = FeedStats()

/// Snapshot for the producer and the engine. The pump thread is the only writer, and both readers
/// run on it (the deferred-cut log site and the pump-finished handler).
var feedStats: FeedStats { stats }

/// One-shot: a bridge that stays silent for an hour costs one line, not one per packet.
private var silentFeedReported = false

/// Enough source that no start-up latency explains the silence. The largest encoder frame the
/// bridge builds is 1536 samples (E-AC-3) and the smallest source packet it is fed is 512 (DTS),
/// so three packets is the honest worst case before the first output and 64 is two orders past it.
private static let silentFeedPacketThreshold = 64

/// Snapshot of bytes live in the bridge's growable buffers, for the engine memory probe. Both fields grow on
/// the FFmpeg side (FIFO reallocs upward, swr delay buffer reallocates on rate/layout shift), so a
/// monotonically rising value points here vs the segment muxer or HLS server. Costs: two C calls, no allocations.
Expand Down Expand Up @@ -641,6 +716,8 @@ final class AudioBridge: @unchecked Sendable {
return []
}

stats.packetsFed += 1

var results: [UnsafeMutablePointer<AVPacket>] = []

// Capture packet.pts for the encoder-PTS rebase, NOT the decoded frame's pts. Issue #7: for codecs with
Expand All @@ -658,7 +735,20 @@ final class AudioBridge: @unchecked Sendable {
// Drain every decodable frame into the FIFO. The PTS rebase fires on the first frame after a segment
// boundary so FLAC timestamps track the source rather than drifting on FIFO leftover (uses packetPts, not sf.pts).
func receiveDecodedFrames() throws {
while avcodec_receive_frame(dec, sf) >= 0 {
while true {
// AE#396: the loop used to be `while avcodec_receive_frame(...) >= 0`, which reads
// "drain what is there" and behaves as "drop the reason there is nothing". A decoder
// that rejects every frame of a stream is then indistinguishable from one that has
// simply caught up, for the whole session.
let receiveRet = avcodec_receive_frame(dec, sf)
if receiveRet < 0 {
if receiveRet != FFmpegErr.eagain, receiveRet != FFmpegErr.eof {
stats.decodeErrors += 1
stats.lastDecodeErrorCode = receiveRet
}
break
}
stats.framesDecoded += 1
if rebaseFromNextSourcePTS, packetPts != Self.avNoPTS {
nextEncoderPTS = av_rescale_q(packetPts, srcTimeBase, encoderTimeBase)
rebaseFromNextSourcePTS = false
Expand All @@ -676,6 +766,8 @@ final class AudioBridge: @unchecked Sendable {
sendRet = avcodec_send_packet(dec, pkt)
}
if sendRet == FFmpegErr.invalidData || sendRet == FFmpegErr.einval {
stats.packetsRejected += 1
stats.lastDecodeErrorCode = sendRet
// Skippable single-packet rejections, decoder stays usable for the next packet:
// invalidData = corrupt source packet (glitchy live MPEG-TS, broken mp2 header);
// einval (-22) = a DTS-HD MA XLL frame that residual-codes channels without a usable core
Expand Down Expand Up @@ -709,9 +801,28 @@ final class AudioBridge: @unchecked Sendable {
throw error
}

noteSilentFeedIfNeeded()
return results
}

/// AE#396: say it once, the moment enough source has gone in for the silence to be structural
/// rather than start-up latency. Without this the only downstream evidence is movenc's -22 on a
/// moov it cannot build, which names the muxer and the source and never the bridge. `EngineLog`
/// has no error level, so the `ERROR:` prefix carries it, as in the #165 cascade line.
private func noteSilentFeedIfNeeded() {
guard !silentFeedReported,
stats.isSilent,
stats.packetsFed >= Self.silentFeedPacketThreshold else { return }
silentFeedReported = true
EngineLog.emit(
"[AudioBridge] ERROR: AE#396 the bridge has produced no encoded audio at all "
+ "(mode=\(mode.rawValue)): \(stats.summary). The mp4 muxer can only build an "
+ "AC-3/E-AC-3 sample entry from a packet that was written, so this session will fail "
+ "its first segment cut unless output starts.",
category: .session
)
}

/// Align swr's INPUT side to the frame the decoder actually produced. libswresample reads `extended_data`
/// planes strictly per its configured input format, so if the decoder emits a different (sample_fmt,
/// sample_rate, ch_layout) than swr was set up for, the bytes are misread: lossless DTS-HD MA decodes to
Expand Down Expand Up @@ -785,17 +896,26 @@ final class AudioBridge: @unchecked Sendable {
// EXC_BAD_ACCESS at 0x0. Skip such frames (the video path tolerates the same corruption).
guard sf.pointee.nb_samples > 0,
let ext = sf.pointee.extended_data,
ext.pointee != nil else { return }
ext.pointee != nil else {
stats.framesDroppedBeforeFIFO += 1
return
}

// Align swr's INPUT to the frame the decoder actually produced before converting. No-op once the seed
// matched (the usual case); only a wrong init seed or a genuine mid-stream format change rebuilds swr.
reconfigureSwrInputIfNeeded(forFrame: sf, enc: enc)
// The rebuild reuses the context pointer on success, but swr_alloc_set_opts2 frees it on a set-opts
// failure (swr_free(ps) -> swrCtx == nil), which would dangle the caller's `swr`. Re-bind to the live one.
guard let swr = swrCtx else { return }
guard let swr = swrCtx else {
stats.framesDroppedBeforeFIFO += 1
return
}

let outNbSamples = swr_get_out_samples(swr, sf.pointee.nb_samples)
guard outNbSamples > 0 else { return }
guard outNbSamples > 0 else {
stats.framesDroppedBeforeFIFO += 1
return
}

let nChannels = enc.pointee.ch_layout.nb_channels
let isPlanar = av_sample_fmt_is_planar(pcmSampleFmt) != 0
Expand Down Expand Up @@ -827,16 +947,20 @@ final class AudioBridge: @unchecked Sendable {
)
}
}
guard producedSamples > 0 else { return }
guard producedSamples > 0 else {
stats.framesDroppedBeforeFIFO += 1
return
}

// av_audio_fifo_write takes void **data; the same array works for both layouts (FIFO knows the format).
_ = outPtrs.withUnsafeMutableBufferPointer { fifoBuf in
let written = outPtrs.withUnsafeMutableBufferPointer { fifoBuf in
fifoBuf.baseAddress!.withMemoryRebound(
to: UnsafeMutableRawPointer?.self, capacity: bufferCount
) { rebound in
av_audio_fifo_write(fifo, rebound, producedSamples)
}
}
if written > 0 { stats.samplesEnqueued += Int64(written) }
}

/// Pull frame_size chunks from the FIFO and encode each. requireFull true stops below frame_size (streaming);
Expand Down Expand Up @@ -904,11 +1028,14 @@ final class AudioBridge: @unchecked Sendable {
break
}
if recvRet < 0 {
stats.encodeErrors += 1
stats.lastEncodeErrorCode = recvRet
var p: UnsafeMutablePointer<AVPacket>? = outPkt
trackedPacketFree(&p)
break
}
outputBytesLifetime += Int64(outPkt.pointee.size)
stats.packetsEmitted += 1
results.append(outPkt)
}
}
Expand Down
7 changes: 7 additions & 0 deletions Sources/AetherEngine/PlaybackErrorInfo.swift
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,13 @@ public struct PlaybackErrorKind: RawRepresentable, Sendable, Equatable, Hashable
public static let liveReloadNeverReady = PlaybackErrorKind(rawValue: "liveReloadNeverReady")
/// An audio-track switch failed; the session it replaced is gone with it.
public static let audioTrackSwitchFailed = PlaybackErrorKind(rawValue: "audioTrackSwitchFailed")
/// A source whose audio has to be transcoded (MP3, MP2, DTS, TrueHD, Vorbis, PCM: anything not
/// legal for stream-copy into fMP4) produced no encoded audio at all, so the mp4 muxer could not
/// build the sample entry it can only derive from a written packet and the first segment cut
/// failed (AE#396). Distinct from `vodSourceFailed`, which this used to arrive as: the source is
/// neither gone nor unreadable, and a second player that decodes the track itself will play it.
/// A host with a fallback ladder should DEMOTE on this one, not end the ladder.
public static let audioBridgeProducedNoOutput = PlaybackErrorKind(rawValue: "audioBridgeProducedNoOutput")
}

/// Machine-readable companion to the text inside `PlaybackState.error` (#376).
Expand Down
30 changes: 25 additions & 5 deletions Sources/AetherEngine/Video/HLSSegmentProducer.swift
Original file line number Diff line number Diff line change
Expand Up @@ -376,6 +376,11 @@ final class HLSSegmentProducer: @unchecked Sendable {
}
/// Set by the host from a previous producer's verdict; skips the search entirely.
private let audioMoovPrimeKnownUnobtainable: Bool

/// AE#396: the bridge's own account of what it did with the source, for the pump-finished handler.
/// Nil for a stream-copy session, which has no bridge and whose analogous verdict is
/// `audioMoovPrimeUnobtainable`.
var audioBridgeFeedStats: AudioBridge.FeedStats? { audioConfig?.bridge?.feedStats }
private var currentMuxer: MP4SegmentMuxer?
private var currentMuxerSegmentIndex: Int = .min

Expand Down Expand Up @@ -1739,11 +1744,26 @@ final class HLSSegmentProducer: @unchecked Sendable {
// order). Nothing was written, so the pump exits to let the host rebuild with a prime frame; the
// scan for that frame happens on the way out.
cutDeferredAwaitingAudioSampleEntry = true
EngineLog.emit(
"[HLSSegmentProducer] AE#222 seg-\(currentMuxerSegmentIndex).m4s cut deferred: audio sample "
+ "entry needs a parsed packet and none has been muxed; scanning forward for a prime frame",
category: .session
)
// AE#396: the two ways to arrive here are not the same defect and must not read the same.
// Stream-copy audio is missing a SOURCE packet, which the prime scan goes looking for. A
// bridged session's muxed frames come from the encoder, so there is nothing in the source
// to scan for and `scanForAudioMoovPrimeFrame` returns without looking: announcing a scan
// there described an action that never happened, on the one path where the interesting
// question ("why has the bridge emitted nothing?") had no line at all.
if let bridge = audioConfig?.bridge {
EngineLog.emit(
"[HLSSegmentProducer] AE#396 seg-\(currentMuxerSegmentIndex).m4s cut deferred: the "
+ "audio sample entry is built from a BRIDGED packet and the bridge has muxed none. "
+ "Bridge: \(bridge.feedStats.summary)",
category: .session
)
} else {
EngineLog.emit(
"[HLSSegmentProducer] AE#222 seg-\(currentMuxerSegmentIndex).m4s cut deferred: audio sample "
+ "entry needs a parsed packet and none has been muxed; scanning forward for a prime frame",
category: .session
)
}
return nil
case .failed:
// Failed cut: muxer has no open staging fd, every byte is silently discarded. Fatal.
Expand Down
Loading
Loading