Improve YouTube transcript paragraph reconstruction

[miguelcorderocollar]

Problem

YouTube transcript segments are often split by timestamp/segment boundaries, not by natural paragraphs. PR #76 added lightweight line-break cleanup for timestamp-free transcripts, but the output can still read like a single long block or preserve awkward boundaries where the transcript segment ended mid-thought.

The hard part is distinguishing:

• a sentence ending inside the same paragraph
• a true paragraph/topic/speaker boundary
• a transcript fragment caused by YouTube segment timing

There likely is not a small dependency that can reliably infer paragraph boundaries from plain text alone. Sentence boundary detection is easier; paragraph reconstruction needs heuristics based on transcript structure.

Proposed direction

Add an optional smarter cleanup mode for YouTube transcripts that reconstructs readable paragraphs using lightweight heuristics instead of adding a heavy NLP dependency.

Signals to consider:

• Preserve chapter headings: ## Chapter title
• Preserve speaker lines: - [Speaker] ...
• Preserve sound/action lines: (music), [applause], etc.
• Merge lines when the previous line does not end with sentence punctuation
• Avoid merging into lines that start with -, [, (, or #
• Optionally insert paragraph breaks every 3–5 complete sentences
• If timestamps are available internally, use timestamp gaps as a stronger paragraph signal, e.g. a complete sentence followed by a 5–10s gap

Possible implementation

Introduce a function such as:

export function reconstructTranscriptParagraphs(segments, options = {}) {
  // options: maxSentencesPerParagraph, paragraphGapSeconds, preserveSpeakerBoundaries, etc.
}

Instead of operating only on the final joined transcript string, this could work on structured transcript segments:

{
  timestamp: '0:05',
  seconds: 5,
  text: 'deep within lorem ipsum,',
  type: 'transcript' | 'chapter'
}

This would make it easier to use timestamp gaps and preserve semantic boundaries.

Acceptance criteria

• Timestamp-free YouTube transcript output is grouped into readable paragraphs.
• Existing cleanUpLineBreaks behavior remains available and backward compatible.
• Chapters, speaker labels, and sound/action lines are preserved as separate boundaries.
• Tests cover:
  • fragmented lines that should merge
  • completed sentences that should remain in the same paragraph
  • speaker boundary preservation
  • chapter boundary preservation
  • paragraph split after configurable sentence count
  • paragraph split after timestamp gap, if timestamps are used internally
• No heavy NLP dependency unless there is a clear, measured benefit.

Notes

Useful lightweight options to evaluate:

• Native Intl.Segmenter for sentence segmentation where available
• sentence-splitter or sbd only if native/browser behavior is insufficient

The likely best approach for ExtractMD is: punctuation + transcript markers + chapters + optional timestamp gaps.