Gling.ai Local Clone

This repository provides a local-first replica of the core productivity features offered by gling.ai: automatic transcription, filler-word and silence removal, LLM-assisted script polish, and b-roll cue suggestions. Everything runs on your workstation so you can keep creative workflows private.

Features

• High-quality transcription via Deepgram Nova (smart format + filler capture), AssemblyAI (cloud diarization + formatting), or local faster-whisper, with optional OpenAI fallback.
• Intelligent cut detection that combines filler-word patterns and adaptive silence removal.
• Automatic render using ffmpeg with configurable fade, crossfade, and smart policies plus optional loudness matching.
• Creative assists: AI-generated title ideas and b-roll search prompts with optional Pexels lookup.
• GPT-5 finishing editor (low reasoning mode) that flags redundant stretches, trims pacing issues, and makes professional-grade cut decisions while preserving transcript accuracy.
• LLM-guided cut refinement plus full per-run logging (runs/<id>/artifacts/pipeline.log) capturing transcription stats, AI prompts/responses, edited script, and render steps.
• Full per-run logging (written to runs/<id>/artifacts/pipeline.log) capturing transcription stats, heuristic decisions, LLM prompts/responses, and render steps.
• CLI and FastAPI services, plus dedicated audio QA (python -m cli.audio_qa) and benchmarking (python -m cli.benchmark_transitions) utilities.

Quickstart

Prerequisites

• Python 3.10+
• ffmpeg installed and available on $PATH.
• (Optional) GPU for faster Whisper inference.

Installation

python -m venv .venv
source .venv/bin/activate
pip install -e .
cp .env.example .env  # then edit with your keys/preferences

Environment

Set environment variables to select providers:

export GLING_TRANSCRIPTION_PROVIDER=assemblyai       # set deepgram_nova, faster_whisper, or openai_whisper as needed
# Deepgram-specific overrides (only used when provider=deepgram_nova)
export DEEPGRAM_API_KEY=dg-...                       # required for Deepgram Nova
export GLING_DEEPGRAM_MODEL=nova-3                   # pick nova-2, nova-3, or domain-specific models
export GLING_DEEPGRAM_SMART_FORMAT=true              # auto casing + punctuation
export GLING_DEEPGRAM_PUNCTUATE=true                 # only used when smart_format=false
export GLING_DEEPGRAM_FILLER_WORDS=true              # surface "uh/um" tokens for heuristics
export GLING_DEEPGRAM_UTTERANCES=true                # request Deepgram utterance segmentation
export GLING_DEEPGRAM_UTT_SPLIT=0.8                  # silence (s) that defines a new utterance
export GLING_DEEPGRAM_UTTERANCE_WINDOW_MIN_DURATION=1.0  # minimum utterance length passed to LLM
export GLING_WHISPER_MODEL=medium                    # still used if you switch back to faster_whisper
export OPENAI_API_KEY=sk-...                         # if using OpenAI LLMs
export GLING_LLM_PROVIDER=openai                   # enable GPT-5 editing
export GLING_LLM_MODEL=gpt-5                      # default model
export GLING_LLM_CUT_REFINE=true                   # keep LLM guardrails on
export GLING_LLM_TRANSCRIPT_MAX_CHARS=0            # 0 = send full transcript to GPT-5
export GLING_LLM_REASONING_EFFORT=low              # leverage GPT-5 low reasoning mode
export GLING_LLM_VERBOSITY=low                     # concise JSON responses
export GLING_RENDER_VIDEO_CODEC=libx264            # rendering codec (filters require re-encode)
export GLING_RENDER_VIDEO_CRF=14                   # lower CRF = higher quality (default 14)
export GLING_RENDER_VIDEO_PRESET=slow              # x264 preset for higher quality / compression
export GLING_RENDER_AUDIO_CODEC=aac                # audio codec after fades
export GLING_RENDER_AUDIO_BITRATE=192k             # target audio bitrate
export GLING_RENDER_PIX_FMT=yuv420p                # ensure broad compatibility
export GLING_LLM_CUT_MAX_DURATION=15               # longest allowed removal
export GLING_LLM_CUT_EXTEND_SEC=1.0                # window padding for cuts
export GLING_LOG_LEVEL=DEBUG                       # capture detailed traces
export GLING_LOG_TO_STDOUT=true                    # mirror logs in console
export PEXELS_API_KEY=your-key                     # optional b-roll

# AssemblyAI-specific overrides (only used when provider=assemblyai)
export ASSEMBLYAI_API_KEY=your-assemblyai-key
export GLING_ASSEMBLYAI_MODEL=best                 # best, nano, or any supported AssemblyAI model
export GLING_ASSEMBLYAI_SPEAKER_LABELS=true        # enable diarization metadata
export GLING_ASSEMBLYAI_FORMAT_TEXT=true           # smart casing + punctuation
export GLING_ASSEMBLYAI_PUNCTUATE=true             # fallback punctuation toggle
export GLING_ASSEMBLYAI_DISFLUENCIES=true          # include filler/disfluency tokens
export GLING_ASSEMBLYAI_LANGUAGE_DETECTION=true    # auto-detect source language when --language not provided

Run via CLI

python -m cli.glcli /path/to/video.mp4 --language en --smart

Switch --smart for --fade or --crossfade to select the audio transition policy without touching environment variables. Append --no-llm to skip GPT cut refinement and keep heuristic cuts only. Outputs include transcript JSON, cut list, rendered video, a transition plan, and metadata under runs/<run-id>/. AI rewrite + cut reasoning land in artifacts/clean_script.txt, artifacts/llm_refine.json, the run log, and artifacts/transition_plan.json chronicles the per-cut silence analysis.

Run via API

Start the server:

uvicorn api.main:app --reload

The REST surface is versioned under /api/v1:

• Submit a run

  curl -X POST http://localhost:8000/api/v1/runs \
       -H "Content-Type: application/json" \
       -d '{
            "video_path": "/abs/path/video.mp4",
            "language": "en",
            "render": true,
            "generate_titles": true,
            "suggest_broll": false,
            "transition_mode": "smart",
            "llm_refine": true
          }'

  Response (202 Accepted) returns the run identifier, initial status, and hyperlinks.

  Minimal run with titles/b-roll disabled and fade-only audio transitions:

  curl -X POST http://localhost:8000/api/v1/runs \
       -H "Content-Type: application/json" \
           -d '{
             "video_path": "/Volumes/ExternalSSD/video-recordings/app.mp4",
             "render": true,
             "generate_titles": false,
             "suggest_broll": false,
             "transition_mode": "fade",
             "llm_refine": false
           }'

• List runs

  curl 'http://localhost:8000/api/v1/runs?page=1&page_size=20'

• Inspect a run

  curl http://localhost:8000/api/v1/runs/<run_id>

• Download artifacts

  curl -L http://localhost:8000/api/v1/runs/<run_id>/artifacts/cuts -o cuts.json
  curl -L http://localhost:8000/api/v1/runs/<run_id>/transcript -o transcript.json

  Supported artifact names: transcript, cuts, metadata, render, log, clean_script, llm_refine, transition_plan, qa_report, llm_metrics.

• Health check

  curl http://localhost:8000/api/v1/health

How to Run Audio QA

1. Produce a run so we have keep_segments metadata (adjust flags as needed):

   python3.10 -m cli.glcli /Users/lucifermornens/Downloads/wcut_ai/kickstarter.mp4 --language en --smart

   The CLI prints the run id; metadata will land under runs/<run-id>/artifacts/metadata.json.

2. Render fade, crossfade, and smart variants for review:

python3.10 -m cli.audio_qa /Users/lucifermornens/Downloads/wcut_ai/kickstarter.mp4 \
  --metadata runs/<run-id>/artifacts/metadata.json \
  --mode fade --mode crossfade --mode smart \
  --out /Users/lucifermornens/Downloads/wcut_ai/runs/qa_kickstarter \
  --loudness-match --fade 0.12 --crossfade 0.18 --curve sine

The QA command now also writes transition_plan.json in the output directory, capturing per-cut silence readings, fade lengths, and transcript snippets so you can spot risky overlaps quickly.

Project Layout

• docs/architecture.md – system design and extension points.
• gling_local/ – core pipeline modules (audio, transcription, edit logic, LLM, b-roll, orchestration).
• cli/glcli.py – command-line interface.
• cli/audio_qa.py – headphone-first QA harness that renders fade/crossfade/smart variants and outputs a checklist.
• cli/benchmark_transitions.py – synthetic fixture generator and transition benchmark.
• api/main.py – FastAPI service.

Next Steps

• Add diarization (speaker detection) to improve dialogue-heavy videos.
• Train statistical filler detectors with logistic regression on word embeddings.
• Extend renderer with auto-punch-in (zoom) filters for visual variety.
• Ship a lightweight React front-end to browse runs and approve edits.
• Extend audio QA tooling with perceptual-weighted metering and anomaly reports.

Testing

pip install .[dev]
pytest -m "not integration"
pytest -m integration  # optional, requires ffmpeg and ~30s runtime