diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml new file mode 100644 index 0000000..4a5e0b5 --- /dev/null +++ b/.github/workflows/ci.yml @@ -0,0 +1,26 @@ +name: ci + +on: + push: + pull_request: + +jobs: + test: + runs-on: windows-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 + with: + python-version: '3.12' + cache: pip + - name: Install test dependencies + shell: pwsh + run: | + python -m pip install --upgrade pip + python -m pip install -r requirements.txt + - name: Compile + shell: pwsh + run: python -m compileall app tests launch_seestory.pyw stop_seestory.py shutdown_diagnostic.py + - name: Unit tests + shell: pwsh + run: python -m unittest discover -s tests -v diff --git a/.gitignore b/.gitignore index 78ffb0b..c1e626c 100644 --- a/.gitignore +++ b/.gitignore @@ -1,25 +1,31 @@ # Python venv/ +.venv/ __pycache__/ -*.pyc +*.py[cod] # Runtime / generated +logs/ +runtime/ seestory.log +SeeStory-shutdown-diagnostic-*.txt output/* !output/.gitkeep uploads/* !uploads/.gitkeep -# Copilot sign-in session (never commit credentials) -session/ +# Local Windows launchers and shortcuts (documented in BUILD.md) +*.bat +*.cmd +*.lnk -# Optional Copilot library — not committed; add it locally (see BUILD.md) -Windows-Copilot-API/ +# Local model/download caches +.cache/ +models/ -# Windows launchers are generated locally — see BUILD.md -*.bat -screenshots/header.png -screenshots/setup-photoreal.png -screenshots/storyboard.png -screenshots/style-storybook.png -screenshots/header.png +# Editor / OS +.vscode/ +.idea/ +.DS_Store +Thumbs.db +desktop.ini diff --git a/BUILD.md b/BUILD.md index 2d05585..e6078f0 100644 --- a/BUILD.md +++ b/BUILD.md @@ -1,477 +1,342 @@ -# SeeStory — Launcher Build Guide +# SeeStory build and Windows launcher guide -The Windows `.bat` launchers are intentionally **kept out of version control** (they're in `.gitignore`). This guide documents every launcher and gives its full contents so you can recreate them after cloning. +This document is the source of truth for rebuilding the Windows helper files that are intentionally excluded from git. The release/test ZIP may contain these BAT files for convenience, but the repository keeps `*.bat`, `*.cmd`, and `*.lnk` ignored. -## Table of Contents +## Build/runtime layout -- [Why the launchers aren't in the repo](#why-the-launchers-arent-in-the-repo) -- [How to recreate them](#how-to-recreate-them) -- [The launchers at a glance](#the-launchers-at-a-glance) -- [Optional: the Copilot library](#optional-the-copilot-library) -- [File contents](#file-contents) - - [`setup.bat`](#setupbat) - - [`run.bat`](#runbat) - - [`stop.bat`](#stopbat) - - [`check_gpu.bat`](#check-gpubat) - - [`install_stable_diffusion.bat`](#install-stable-diffusionbat) - - [`setup_copilot.bat`](#setup-copilotbat) - - [`login_copilot.bat`](#login-copilotbat) -- [Notes (encoding & line endings)](#notes-encoding--line-endings) +- `launch_seestory.pyw` — hidden desktop launcher. Starts Flask with `pythonw.exe`, waits for readiness, opens the tracked Chrome/Edge app window, and shuts the server down when that window closes. +- `app/desktop_runtime.py` — lifecycle heartbeat, Windows HIGH priority/power-throttling safeguards, diagnostics state, and system monitor API. +- `stop_seestory.py` — explicit stop helper used by `stop.bat`. +- `shutdown_diagnostic.py` — writes a timestamped shutdown/process diagnostic. +- `install_models.py` — installer-only model download, warm-load, CUDA inference smoke test, and model-install logging. +- `build_icons.py` — recreates the high-resolution PNG and multi-size Windows ICO used by the shortcut. +- `assets/SeeStory.ico` — desktop shortcut icon. +- `output/`, `uploads/`, `logs/`, `runtime/`, `venv/` — local/generated state and never release-source content. -## Why the launchers aren't in the repo +## Local image models -They're small, machine‑specific conveniences rather than application code, and downloaded `.bat` files can trip Windows SmartScreen/antivirus — so they're regenerated locally instead of committed. The application itself lives in `app/`; the launchers only wrap `python -m app.server` and the setup steps. +The app has one local image-generation path with automatic style routing: Cinematic/Storybook/Noir/Oil/Ink use `Lykon/dreamshaper-xl-lightning`, and Photorealistic uses `SG161222/RealVisXL_V5.0_Lightning`. Both repositories publish a complete Diffusers fp16/Safetensors layout. +The weights are not committed or bundled; each downloaded model remains under its own upstream license. -## How to recreate them +`install_all.bat` owns every large model transfer. It installs CUDA-enabled PyTorch and the image stack, downloads/resumes the required illustration model, warm-loads it, performs a real CUDA inference, records peak VRAM, and saves a smoke-test image. The optional Photorealistic model is offered and verified the same way. The desktop runtime is forced offline for Hugging Face/Transformers, so Flask never starts model downloads. ffmpeg is a separate system dependency and is checked by the installer. Pass `default` to skip the optional-model prompt or `all` to install both models non-interactively. -For each file in [File contents](#file-contents): +## Desktop startup and shutdown -1. Create a new text file in the SeeStory folder with the **exact** name (e.g. `setup.bat`). -2. Paste the matching block below. -3. Save it with **Windows (CRLF) line endings** and ANSI/UTF‑8 encoding. -4. Double‑click `setup.bat` first, then `run.bat`. +The launcher reuses `%LOCALAPPDATA%\SeeStory\BrowserProfile` for the dedicated browser app. Do not switch back to a unique profile per launch: fresh browser profiles can trigger first-run/profile process churn and visible window flashes. The profile is outside the repository. A Windows named mutex also prevents multiple launcher instances from racing if the shortcut is double-clicked more than once. -> Tip: in Notepad, *Save As* → set *Encoding* to ANSI; CRLF is the Windows default. In VS Code, click the `LF`/`CRLF` indicator in the status bar and choose **CRLF**. +Closing the tracked app window calls the token-protected local shutdown endpoint. A browser heartbeat is a fallback for crash/process handoff cases. The explicit `stop.bat` path remains available. -## The launchers at a glance +## Repository rules -| File | What it does | When you run it | -|------|--------------|-----------------| -| `setup.bat` | First‑time setup: builds the Python venv, installs the app + Stable Diffusion, and installs the optional Copilot dependencies. Run once. `setup.bat nosd` skips the big SD download. | once, first | -| `run.bat` | Starts SeeStory and opens Chrome at http://127.0.0.1:5001. Run every time you use it; keep the window open while generating. | every time | -| `stop.bat` | Stops a running SeeStory server and frees port 5001. | as needed | -| `check_gpu.bat` | Prints your PyTorch version and whether the CUDA GPU is detected. | as needed | -| `install_stable_diffusion.bat` | (Re)installs the Stable Diffusion backend on its own. `install_stable_diffusion.bat force` does a clean PyTorch reinstall. | as needed | -| `setup_copilot.bat` | Optional. Installs the Copilot library's dependencies and runs the one‑time Microsoft sign‑in. | once (optional) | -| `login_copilot.bat` | Optional. Re‑runs the Copilot Microsoft sign‑in (refreshes the session). | as needed (optional) | +Before releasing, `git status` should contain no BAT/CMD/shortcut, venv, model cache, runtime state, logs, uploads, generated project output, or shutdown-diagnostic files. `.gitignore` enforces these rules. -## Optional: the Copilot library +## Exact Windows BAT files -**Copilot is entirely optional** — SeeStory runs fully on Stable Diffusion and -placeholders without it. The `Windows-Copilot-API` library it needs is **not -committed to this repo** (it's a third‑party project, kept out of version control -just like the launchers), so a fresh clone won't have it. Add it only if you want -the premium Copilot image source. +The following blocks are generated from the BAT files included in this package. When changing a BAT, update this document in the same release. -To enable Copilot: - -1. Get the library into a `Windows-Copilot-API` folder in the SeeStory root - (next to `app/`): - - ```bat - git clone https://github.com/sums001/Windows-Copilot-API - ``` - - (or download the repo ZIP and extract it so you have - `SeeStory\Windows-Copilot-API\copilot\…`). -2. Build `setup_copilot.bat` and `login_copilot.bat` from - [File contents](#file-contents) if you don't already have them. -3. Run **`setup_copilot.bat`** once — it installs the library's Python - dependencies and the Playwright Chromium used for sign‑in, then opens a - one‑time Microsoft sign‑in. -4. Later, **`login_copilot.bat`** re‑runs that sign‑in to refresh the session. - -After signing in, restart SeeStory; the **copilot** badge turns green and the -**Test Copilot** button confirms it works. If Microsoft changes their protocol -and you hit an "invalid‑event" error, replace the `Windows-Copilot-API\copilot` -folder with the latest from the repo above. - -> The `setup.bat` launcher also tries to install the Copilot dependencies if the -> `Windows-Copilot-API` folder is present; if it's absent, that step is skipped -> harmlessly and SeeStory still installs normally. - -## File contents - -### `setup.bat` - -First‑time setup: builds the Python venv, installs the app + Stable Diffusion, and installs the optional Copilot dependencies. Run once. `setup.bat nosd` skips the big SD download. +### `install_all.bat` ```bat @echo off -setlocal enableextensions +setlocal EnableExtensions cd /d "%~dp0" -title SeeStory setup +title SeeStory - Install All + +set "PYLAUNCH=" +where py >nul 2>nul +if not errorlevel 1 ( + py -3.12 -c "import sys; sys.exit(0 if sys.version_info[:2]==(3,12) else 1)" >nul 2>nul + if not errorlevel 1 set "PYLAUNCH=py -3.12" +) +if not defined PYLAUNCH ( + where python >nul 2>nul + if not errorlevel 1 ( + python -c "import sys; sys.exit(0 if sys.version_info[:2]==(3,12) else 1)" >nul 2>nul + if not errorlevel 1 set "PYLAUNCH=python" + ) +) + +set "HF_HUB_DISABLE_TELEMETRY=1" +set "HF_HUB_DOWNLOAD_TIMEOUT=900" +set "HF_HUB_ETAG_TIMEOUT=60" +set "TOKENIZERS_PARALLELISM=false" +set "PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True" + +if not exist logs mkdir logs +echo ================================================================ +echo SEESTORY - INSTALL ALL +echo ================================================================ echo. -echo ===== SeeStory setup ===== +echo This creates the local environment, installs and GPU-tests the +echo required image model, checks ffmpeg, and creates the shortcut. +echo. +echo IMPORTANT: The required model is about 7 GB. It is downloaded here, +echo not later from the SeeStory web page. Windows Defender may scan the +echo new files during this installation, so the model step can take time. echo. -powershell -NoProfile -Command "Set-ExecutionPolicy -Scope CurrentUser RemoteSigned -Force" >nul 2>nul +if not defined PYLAUNCH ( + echo Python 3.12 was not found. + echo Install Python 3.12 from python.org, then run this file again. + goto :fail +) -REM --- find Python (prefer 3.12 via the launcher, else plain python) -------- -set "PYLAUNCH=python" -where py >nul 2>nul -if not errorlevel 1 set "PYLAUNCH=py -3.12" -echo Using launcher: %PYLAUNCH% -echo. +if exist "venv\Scripts\python.exe" ( + "venv\Scripts\python.exe" -c "import sys; sys.exit(0 if sys.version_info[:2]==(3,12) else 1)" >nul 2>nul + if errorlevel 1 ( + echo [1/8] Existing venv uses the wrong Python version - rebuilding it... + rmdir /s /q venv + ) +) +if not exist "venv\Scripts\python.exe" ( + echo [1/8] Creating Python 3.12 virtual environment... + %PYLAUNCH% -m venv venv + if errorlevel 1 goto :fail +) else ( + echo [1/8] Existing Python 3.12 virtual environment found - reusing it. +) +set "PY=venv\Scripts\python.exe" -REM --- create the venv (skip if present) ----------------------------------- -if exist "venv\Scripts\python.exe" goto :have_venv -echo Creating virtual environment... -%PYLAUNCH% -m venv venv -if errorlevel 1 goto :venv_fail -goto :venv_ok -:venv_fail echo. -echo ERROR: could not create the virtual environment. -echo Install Python 3.10 or newer first: https://www.python.org/downloads/ -goto :end -:have_venv -echo Virtual environment already exists - reusing it. -:venv_ok -set "VPY=venv\Scripts\python.exe" +echo [2/8] Updating pip and installing core requirements... +"%PY%" -m pip install --upgrade pip setuptools wheel +if errorlevel 1 goto :fail +"%PY%" -m pip install -r requirements.txt +if errorlevel 1 goto :fail echo. -echo Upgrading pip... -"%VPY%" -m pip install --upgrade pip +echo [3/8] Checking NVIDIA PyTorch / CUDA... +"%PY%" -c "import torch,sys; ok=torch.cuda.is_available(); print('Current PyTorch:',torch.__version__); print('CUDA available:',ok); sys.exit(0 if ok else 1)" 2>nul +if errorlevel 1 ( + echo Installing CUDA 12.8 PyTorch for RTX 50-series and other NVIDIA GPUs... + "%PY%" -m pip uninstall -y torch torchvision torchaudio >nul 2>nul + "%PY%" -m pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128 + if errorlevel 1 goto :fail +) else ( + echo CUDA-enabled PyTorch is already working. +) echo. -echo Installing core requirements... -"%VPY%" -m pip install -r requirements.txt -if errorlevel 1 goto :core_fail -goto :core_ok -:core_fail -echo. -echo ERROR: core requirements failed to install. Scroll up to see why. -goto :end -:core_ok +echo [4/8] Verifying the GPU image-generation stack... +"%PY%" -c "import torch,diffusers,transformers,accelerate,safetensors,sys; ok=torch.cuda.is_available(); print('PyTorch:',torch.__version__); print('Diffusers:',diffusers.__version__); print('Transformers:',transformers.__version__); print('CUDA:',ok); print('GPU:',torch.cuda.get_device_name(0) if ok else 'NOT AVAILABLE'); sys.exit(0 if ok else 1)" +if errorlevel 1 goto :gpu_fail -if /I "%~1"=="nosd" goto :skip_sd - -echo. -echo ============================================================ -echo Installing Stable Diffusion - local image generation. -echo Large download, can take several minutes. To skip it, -echo close this window and run: setup.bat nosd -echo ============================================================ echo. -echo [1/2] Checking PyTorch / GPU... -"%VPY%" -c "import torch,sys; sys.exit(0 if torch.cuda.is_available() else 1)" 2>nul -if not errorlevel 1 goto :torch_have -echo Installing PyTorch with CUDA GPU support. Only 'torch' is replaced - -echo shared packages like jinja2 and MarkupSafe are left untouched. -"%VPY%" -m pip uninstall -y torch >nul 2>nul -"%VPY%" -m pip install torch --index-url https://download.pytorch.org/whl/cu128 -if not errorlevel 1 goto :torch_done -echo CUDA 12.8 unavailable - trying CUDA 12.4... -"%VPY%" -m pip install torch --index-url https://download.pytorch.org/whl/cu124 -if not errorlevel 1 goto :torch_done -echo No GPU build available - installing CPU-only PyTorch ^(slower^). -"%VPY%" -m pip install torch -goto :torch_done -:torch_have -echo PyTorch with a working CUDA GPU is already installed - skipping the download. -:torch_done +echo [5/8] Checking ffmpeg... +where ffmpeg >nul 2>nul +if errorlevel 1 ( + echo ffmpeg was not found. Attempting a winget install... + where winget >nul 2>nul + if not errorlevel 1 winget install --id Gyan.FFmpeg --exact --accept-package-agreements --accept-source-agreements + where ffmpeg >nul 2>nul + if errorlevel 1 echo WARNING: ffmpeg is still missing. Install it, then restart Windows or sign out/in. +) else ( + ffmpeg -version | findstr /B /C:"ffmpeg version" +) echo. -echo [2/2] diffusers stack... -"%VPY%" -m pip install diffusers transformers accelerate safetensors -if errorlevel 1 goto :sd_warn +echo [6/8] Installing and GPU-testing the required illustration model... +"%PY%" "install_models.py" --model default +if errorlevel 1 goto :model_fail echo. -echo Verifying GPU... -"%VPY%" -c "import torch; ok=torch.cuda.is_available(); print(' PyTorch', torch.__version__); print(' GPU available:', ok); print(' GPU:', torch.cuda.get_device_name(0) if ok else 'CPU only')" -goto :sd_done +echo [7/8] Optional Photorealistic model... +set "INSTALL_PHOTO=" +if /I "%~1"=="all" set "INSTALL_PHOTO=Y" +if /I "%~1"=="default" set "INSTALL_PHOTO=N" +if not defined INSTALL_PHOTO ( + choice /C YN /N /M "Install and GPU-test the optional Photorealistic model too? [Y/N] " + if errorlevel 2 (set "INSTALL_PHOTO=N") else (set "INSTALL_PHOTO=Y") +) +if /I "%INSTALL_PHOTO%"=="Y" ( + "%PY%" "install_models.py" --model photoreal + if errorlevel 1 goto :model_fail +) else ( + echo Skipped. Run install_all.bat again and choose Yes before using Photorealistic. +) -:sd_warn echo. -echo NOTE: Stable Diffusion did not fully install. SeeStory still runs with -echo placeholder frames and the Copilot backend. Retry any time with: -echo install_stable_diffusion.bat -goto :sd_done +echo [8/8] Creating app icons and desktop shortcut... +"%PY%" "build_icons.py" --quiet +if errorlevel 1 echo WARNING: App icon generation failed. +if not exist "assets\SeeStory.ico" goto :icon_missing +powershell -NoProfile -ExecutionPolicy Bypass -Command "$root=(Resolve-Path '.').Path; $desktop=[Environment]::GetFolderPath('Desktop'); $w=New-Object -ComObject WScript.Shell; $s=$w.CreateShortcut((Join-Path $desktop 'SeeStory.lnk')); $s.TargetPath=(Join-Path $root 'venv\Scripts\pythonw.exe'); $s.Arguments='\"'+(Join-Path $root 'launch_seestory.pyw')+'\"'; $s.WorkingDirectory=$root; $s.IconLocation=(Join-Path $root 'assets\SeeStory.ico')+',0'; $s.Description='SeeStory - bring narrated audiobooks to life'; $s.Save()" +if errorlevel 1 echo WARNING: The desktop shortcut could not be created. run.bat still works. +goto :complete -:skip_sd -echo. -echo Skipped Stable Diffusion. SeeStory will use placeholder frames until you -echo install it - run install_stable_diffusion.bat whenever you like. +:icon_missing +echo WARNING: assets\SeeStory.ico is missing. Shortcut was not created. -:sd_done +:complete echo. -echo ============================================================ -echo Copilot backend (optional) - premium images for big moments. -echo The Windows-Copilot-API library is BUNDLED with SeeStory; -echo installing its Python dependencies now... -echo ============================================================ -"%VPY%" -m pip install -r "Windows-Copilot-API\requirements.txt" -if errorlevel 1 goto :copilot_warn +echo ================================================================ +echo INSTALL COMPLETE +echo ================================================================ +echo The required local model is downloaded, warm-loaded, and GPU-tested. +if /I "%INSTALL_PHOTO%"=="Y" echo The optional Photorealistic model also passed its GPU test. +echo Start SeeStory with the desktop shortcut or run.bat. +echo The app will not download model weights while it is running. echo. -echo Installing the Playwright browser used for the one-time sign-in... -"%VPY%" -m playwright install chromium -if errorlevel 1 goto :copilot_warn -goto :copilot_done -:copilot_warn +echo Model install log: logs\model-install.log +echo Smoke images: logs\model-smoke-*.jpg +echo Shutdown problems: run shutdown_diagnostic.bat +echo ================================================================ echo. -echo NOTE: Copilot dependencies did not fully install. SeeStory still runs -echo with Stable Diffusion / placeholder. Retry later with setup_copilot.bat. -:copilot_done +pause +exit /b 0 +:gpu_fail echo. -echo ============================================================ -echo Setup complete! +echo CUDA could not be proven after installation. SeeStory will not silently +echo use CPU for the image model. Update the NVIDIA driver and rerun this file. +goto :fail + +:model_fail echo. -echo Start SeeStory: double-click run.bat -echo opens Chrome at http://127.0.0.1:5001 +echo The model download, warm-load, or real GPU inference test failed. +echo Review logs\model-install.log, then rerun install_all.bat to resume. +goto :fail + +:fail echo. -echo OPTIONAL - Copilot backend (premium images for the big moments): -echo The library is bundled and its dependencies are installed. -echo To enable it, sign in once: double-click login_copilot.bat -echo Then restart SeeStory - the "copilot" badge turns green, and you can -echo click "Test Copilot" in the header to confirm it works. -echo Repo: https://github.com/sums001/Windows-Copilot-API -echo ============================================================ - -:end +echo ================================================================ +echo INSTALL FAILED +echo ================================================================ +echo Review the error above. Nothing was committed or uploaded. echo. pause +exit /b 1 ``` -### `run.bat` - -Starts SeeStory and opens Chrome at http://127.0.0.1:5001. Run every time you use it; keep the window open while generating. +### `setup.bat` ```bat @echo off -REM ============================================================ -REM SeeStory - start the app (VISIBLE console, foreground) -REM Double-click to launch. Chrome opens at http://127.0.0.1:5001. -REM Keep this window open while using SeeStory. Output is also -REM saved to seestory.log. -REM ============================================================ cd /d "%~dp0" -title SeeStory - -if not exist "venv\Scripts\python.exe" ( - echo. & echo No virtual environment found. Please run setup.bat first. & echo. - pause & exit /b 1 -) - -REM Disable console QuickEdit so clicking the window doesn't pause GPU work. -powershell -NoProfile -Command "$sig='[DllImport(\"kernel32.dll\")]public static extern IntPtr GetStdHandle(int h);[DllImport(\"kernel32.dll\")]public static extern bool GetConsoleMode(IntPtr h,out uint m);[DllImport(\"kernel32.dll\")]public static extern bool SetConsoleMode(IntPtr h,uint m);'; $t=Add-Type -MemberDefinition $sig -Name K -Namespace W -PassThru; $h=$t::GetStdHandle(-10); $m=0; [void]$t::GetConsoleMode($h,[ref]$m); [void]$t::SetConsoleMode($h, ($m -bor 0x0080) -band (-bnot 0x0040))" 2>nul - -echo Starting SeeStory... (Chrome will open shortly) -echo. -echo Keep this window open while generating. For full GPU speed, -echo keep it in the foreground. Watch progress in the browser. -echo A copy of all messages is saved to seestory.log -echo. -powershell -NoProfile -Command "$host.UI.RawUI.WindowTitle='SeeStory'; & '%CD%\venv\Scripts\python.exe' -m app.server 2>&1 | Tee-Object -FilePath '%CD%\seestory.log'" - -echo. -echo ============================================================ -echo SeeStory has stopped. If unexpected, see seestory.log -echo ============================================================ -pause +call install_all.bat %* ``` -### `stop.bat` - -Stops a running SeeStory server and frees port 5001. +### `run.bat` ```bat @echo off -REM Stops any running SeeStory server (frees port 5001). -title SeeStory - stop -echo Stopping SeeStory... -for /f "tokens=5" %%P in ('netstat -ano ^| findstr ":5001" ^| findstr LISTENING') do ( - echo killing PID %%P - taskkill /PID %%P /F >nul 2>nul +setlocal +cd /d "%~dp0" +if not exist "venv\Scripts\pythonw.exe" ( + echo SeeStory is not installed yet. + echo Run install_all.bat first. + pause + exit /b 1 ) -echo Done. -timeout /t 2 >nul +start "" /b "venv\Scripts\pythonw.exe" "launch_seestory.pyw" +exit /b 0 ``` -### `check_gpu.bat` - -Prints your PyTorch version and whether the CUDA GPU is detected. +### `stop.bat` ```bat @echo off +setlocal cd /d "%~dp0" -if not exist "venv\Scripts\python.exe" ( echo Run setup.bat first. & pause & exit /b 1 ) -"venv\Scripts\python.exe" -c "import torch; ok=torch.cuda.is_available(); print('PyTorch', torch.__version__); print('GPU available:', ok); print('GPU:', torch.cuda.get_device_name(0) if ok else '(CPU only)')" 2>nul || echo Stable Diffusion not installed yet ^(placeholder + Copilot still work^). -pause +if exist "venv\Scripts\python.exe" ( + "venv\Scripts\python.exe" "stop_seestory.py" +) else ( + py -3.12 "stop_seestory.py" 2>nul || python "stop_seestory.py" +) +timeout /t 2 >nul ``` -### `install_stable_diffusion.bat` - -(Re)installs the Stable Diffusion backend on its own. `install_stable_diffusion.bat force` does a clean PyTorch reinstall. +### `shutdown_diagnostic.bat` ```bat @echo off -setlocal enableextensions +setlocal cd /d "%~dp0" -title SeeStory - install Stable Diffusion -if not exist "venv\Scripts\python.exe" ( - echo Run setup.bat first to create the environment. - echo. - pause - exit /b 1 +if exist "venv\Scripts\python.exe" ( + "venv\Scripts\python.exe" "shutdown_diagnostic.py" +) else ( + py -3.12 "shutdown_diagnostic.py" 2>nul || python "shutdown_diagnostic.py" ) -set "VPY=venv\Scripts\python.exe" - -if /I "%~1"=="force" goto :torch_force -echo Checking PyTorch / GPU... -"%VPY%" -c "import torch,sys; sys.exit(0 if torch.cuda.is_available() else 1)" 2>nul -if not errorlevel 1 goto :torch_have -echo Installing PyTorch with CUDA GPU support ^(only 'torch' is replaced^)... -"%VPY%" -m pip uninstall -y torch >nul 2>nul -goto :torch_attempt -:torch_force -echo Forcing a clean PyTorch reinstall... -"%VPY%" -m pip install --force-reinstall --no-cache-dir torch --index-url https://download.pytorch.org/whl/cu128 -if not errorlevel 1 goto :torch_done -"%VPY%" -m pip install --force-reinstall --no-cache-dir torch --index-url https://download.pytorch.org/whl/cu124 -if not errorlevel 1 goto :torch_done -"%VPY%" -m pip install torch -goto :torch_done -:torch_attempt -"%VPY%" -m pip install torch --index-url https://download.pytorch.org/whl/cu128 -if not errorlevel 1 goto :torch_done -echo CUDA 12.8 unavailable - trying CUDA 12.4... -"%VPY%" -m pip install torch --index-url https://download.pytorch.org/whl/cu124 -if not errorlevel 1 goto :torch_done -echo Installing CPU-only PyTorch ^(slower^)... -"%VPY%" -m pip install torch -goto :torch_done -:torch_have -echo PyTorch with a working CUDA GPU already present - skipping it. -echo ^(To force a clean reinstall: install_stable_diffusion.bat force^) -:torch_done - -echo. -echo Installing diffusers stack... -"%VPY%" -m pip install diffusers transformers accelerate safetensors - echo. -"%VPY%" -c "import torch; ok=torch.cuda.is_available(); print('PyTorch', torch.__version__); print('GPU available:', ok); print('GPU:', torch.cuda.get_device_name(0) if ok else 'CPU only')" -echo. -echo Done. Start SeeStory with run.bat. pause ``` -### `setup_copilot.bat` - -Optional. Installs the Copilot library's dependencies and runs the one‑time Microsoft sign‑in. +### `check_gpu.bat` ```bat @echo off -setlocal enableextensions +setlocal cd /d "%~dp0" -title SeeStory - set up Copilot backend - -set "REPO=Windows-Copilot-API" - -echo. -echo ===================================================================== -echo SeeStory: Copilot backend setup -echo Adds Microsoft Copilot image generation as the premium source for the -echo standout moments. Optional - SeeStory works fine without it. -echo The library is bundled with SeeStory; this just installs its Python -echo dependencies and signs you in. -echo Repo: https://github.com/sums001/Windows-Copilot-API -echo ===================================================================== -echo. - -if not exist "venv\Scripts\python.exe" goto :no_venv -set "VPY=venv\Scripts\python.exe" - -if exist "%REPO%\copilot" goto :have_repo -echo ERROR: the bundled Windows-Copilot-API folder is missing. Re-extract the -echo SeeStory download (it should contain a "Windows-Copilot-API" folder). -goto :end -:have_repo -echo [1/3] Library found: %CD%\%REPO% -echo. - -echo [2/3] Installing its Python requirements (already-installed ones are skipped)... -"%VPY%" -m pip install -r "%REPO%\requirements.txt" -if errorlevel 1 goto :pip_fail -echo. -echo Installing the Playwright Chromium browser for sign-in (skipped if present)... -"%VPY%" -m playwright install chromium -echo. - -if exist "session\token.json" goto :login_skip -echo [3/3] Microsoft sign-in -echo. -echo A Google Chrome window will open at copilot.microsoft.com. -echo Sign in to your Microsoft account and pass any "verify you're human" -echo check. The window CLOSES BY ITSELF once sign-in is detected - you do -echo not need to press anything here. -echo. -echo Press a key when you're ready to open the sign-in window... -pause >nul -"%VPY%" -m app.copilot_login -goto :ready -:login_skip -echo [3/3] Already signed in - existing session found, skipping sign-in. -echo To sign in again later, run login_copilot.bat -:ready - -echo. -echo ===================================================================== -echo Copilot backend is ready. -echo Start (or restart) SeeStory with run.bat - the "copilot" badge in the -echo header should turn green. Click "Test Copilot" there to confirm. -echo. -echo Remember: Copilot is auto-capped and spaced so your account is never -echo hammered. Choose "Both" or "Copilot only" mode in the app to use it. -echo ===================================================================== -goto :end - -:no_venv -echo Please run setup.bat first to create the SeeStory environment. -goto :end -:pip_fail -echo. -echo ERROR: installing the API's requirements failed. Scroll up for the reason. -goto :end - -:end +if not exist "venv\Scripts\python.exe" ( + echo Run install_all.bat first. + pause + exit /b 1 +) +"venv\Scripts\python.exe" -c "import torch; ok=torch.cuda.is_available(); print('PyTorch:',torch.__version__); print('CUDA available:',ok); print('GPU:',torch.cuda.get_device_name(0) if ok else '(CPU only)'); print('CUDA build:',torch.version.cuda)" echo. +nvidia-smi 2>nul pause ``` -### `login_copilot.bat` - -Optional. Re‑runs the Copilot Microsoft sign‑in (refreshes the session). +### `install_stable_diffusion.bat` ```bat @echo off -setlocal enableextensions +setlocal EnableExtensions cd /d "%~dp0" -title SeeStory - Copilot sign-in - +title SeeStory - Repair Local Models if not exist "venv\Scripts\python.exe" ( - echo Run setup.bat first to create the SeeStory environment. - echo. - pause - exit /b 1 + echo Run install_all.bat first. + pause + exit /b 1 ) -if not exist "Windows-Copilot-API\copilot" ( - echo The bundled Windows-Copilot-API folder is missing - re-extract the - echo SeeStory download, then run setup.bat. - echo. - pause - exit /b 1 +set "PY=venv\Scripts\python.exe" +set "HF_HUB_DISABLE_TELEMETRY=1" +set "HF_HUB_DOWNLOAD_TIMEOUT=900" +set "HF_HUB_ETAG_TIMEOUT=60" +set "TOKENIZERS_PARALLELISM=false" +set "PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True" + +if /I "%~1"=="force" "%PY%" -m pip uninstall -y torch torchvision torchaudio +"%PY%" -c "import torch,sys; sys.exit(0 if torch.cuda.is_available() else 1)" 2>nul +if errorlevel 1 ( + "%PY%" -m pip install --upgrade torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128 + if errorlevel 1 goto :fail ) -echo Making sure the Copilot dependencies are installed... -"venv\Scripts\python.exe" -m pip install -q -r "Windows-Copilot-API\requirements.txt" >nul 2>nul -"venv\Scripts\python.exe" -m playwright install chromium >nul 2>nul - -echo Opening Google Chrome for Microsoft / Copilot sign-in. -echo The window closes by itself once you're signed in - nothing to press here. -echo. -"venv\Scripts\python.exe" -m app.copilot_login +"%PY%" -m pip install -r requirements.txt +if errorlevel 1 goto :fail +"%PY%" -c "import torch,sys; ok=torch.cuda.is_available(); print('PyTorch',torch.__version__); print('CUDA',ok); print('GPU',torch.cuda.get_device_name(0) if ok else 'NOT AVAILABLE'); sys.exit(0 if ok else 1)" +if errorlevel 1 goto :fail + +"%PY%" "install_models.py" --model default +if errorlevel 1 goto :fail +choice /C YN /N /M "Install/repair the optional Photorealistic model too? [Y/N] " +if errorlevel 2 goto :done +"%PY%" "install_models.py" --model photoreal +if errorlevel 1 goto :fail + +:done +echo. +echo Local model repair and GPU verification completed. +echo See logs\model-install.log for details. +pause +exit /b 0 +:fail echo. -echo If SeeStory is open, restart it with run.bat so it picks up the session. +echo Model repair failed. Review logs\model-install.log. pause +exit /b 1 ``` -## Notes (encoding & line endings) +## Validation before release + +Review `CHANGELOG.md`, then run: + +```text +python -m compileall app tests launch_seestory.pyw stop_seestory.py shutdown_diagnostic.py +python -m unittest discover -s tests -v +``` -- **Line endings:** save as **CRLF**. `cmd.exe` mostly tolerates LF, but a few constructs (labels, multi‑line blocks) are happier with CRLF. -- **Encoding:** ANSI or UTF‑8 without BOM. A UTF‑8 BOM can make `cmd` choke on the first line. -- **SmartScreen:** the first time you double‑click a `.bat`, Windows may warn. Choose *More info → Run anyway* (or unblock it in the file's Properties). -- **Paths:** every launcher does `cd /d "%~dp0"`, so they work from wherever the SeeStory folder lives — just keep them in the SeeStory root next to `app/`. +On Windows, also run `check_gpu.bat`, launch from the desktop shortcut, verify a real sample image, build a short MP4, and verify that closing the app window leaves no SeeStory Python process behind. diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..0a06ea2 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,103 @@ +# SeeStory changelog + +## 2026-08-03 — Local GPU desktop release + +This release turns SeeStory into a local-only Windows desktop-style application, removes the retired cloud image experiment, improves model quality and startup behavior, and hardens the complete ebook-to-video workflow. + +### Installation and local models + +- Added `install_all.bat` as the complete Windows installation path. +- Moved all multi-gigabyte model downloads out of the Flask application and into the installer. +- The installer now downloads or resumes the required DreamShaper XL Lightning model, warm-loads it, performs a real CUDA inference, records peak VRAM use, and writes `logs/model-smoke-default.jpg`. +- The optional RealVisXL V5 Lightning Photorealistic model can be installed and GPU-tested during the same installation. +- Added non-interactive installer modes: `install_all.bat default` installs the required model only, while `install_all.bat all` installs both models. +- Added `install_stable_diffusion.bat` as a repair/reverification path for the local model stack. +- Runtime model access is now cache-only. The running app cannot silently begin a large Hugging Face download. +- Missing or incomplete model caches now produce an immediate repair instruction instead of leaving the browser waiting on an unexpected download. +- The installer requires Python 3.12, CUDA-enabled PyTorch, and a real NVIDIA GPU inference. Silent CPU fallback is not accepted. +- Added model-install logging and resumable Hugging Face download settings. + +### Image generation + +- Simplified SeeStory to one local image-generation path with automatic style routing. +- DreamShaper XL Lightning is used for Cinematic, Storybook, Noir, Oil, Ink, and other illustrated styles. +- RealVisXL V5 Lightning is used for Photorealistic output when that optional model is installed. +- Removed the incompatible Juggernaut XL repository route that failed under Safetensors-only Diffusers loading. +- Strengthened human-anatomy, scene-coherence, negative-prompt, malformed-face, duplicate-person, extra-limb, extra-finger, and accidental-text suppression. +- Added model-specific DPM-Solver scheduling and VRAM-aware model CPU offload. +- Retained manual per-scene regeneration because local diffusion models cannot guarantee perfect anatomy on every image. + +### Interface cleanup + +- Removed all references to the retired cloud image provider from the Flask page, application code, requirements, documentation, and tests. +- Removed image-provider/source choices from setup and storyboard cards. +- Removed source badges and the green readiness strip. +- Removed the nonfunctional SeeStory process row from the system monitor. +- The monitor now shows only CPU, system RAM, GPU usage, VRAM, and GPU temperature. +- Removed first-run model-download wording from sample generation. Model installation is an installer responsibility. +- Corrected subtitle upload handling to advertise the `.srt` format the backend actually processes. + +### Windows desktop behavior + +- Added a hidden `pythonw.exe` desktop launcher and a high-resolution SeeStory icon for the desktop shortcut and web page. +- Added `build_icons.py` so repository clones generate the PNG/ICO assets during installation instead of requiring generated binary assets in git. +- SeeStory now opens maximized in a dedicated Chrome or Edge application window. +- Reuses one isolated browser profile instead of creating a new profile on each launch, avoiding first-run browser/profile window bursts. +- Added a Windows named single-instance guard to prevent competing launches after accidental double-clicks. +- Added lightweight startup health checking so opening the window does not initialize the image stack. +- Hidden-process flags are used for ffmpeg, ffprobe, antiword, and other helper processes to prevent console-window flashes. +- Runs the server at Windows HIGH process priority, disables execution-speed/background throttling, and prevents sleep during long jobs. +- Closing the dedicated app window now triggers the same controlled shutdown as `stop.bat`. +- Added browser-process tracking, a token-protected shutdown endpoint, and a heartbeat fallback. +- Added persistent launcher, server, lifecycle, model-install, and shutdown diagnostics under `logs/`. +- Added `shutdown_diagnostic.bat` and `shutdown_diagnostic.py` for cases where a Python process remains after the browser closes. + +### Storyboard and ebook fixes + +- Fixed EPUB extraction that could count wrapper `
` text and its nested paragraphs twice. +- Applied the same nonduplicating block extraction rules to HTML documents. +- Shot text is now non-overlapping and short chapters naturally produce fewer scenes instead of duplicated filler. +- The prompt director tracks recently selected scenes within a chapter and chooses another concrete sentence when a near-identical scene would otherwise repeat. +- Existing saved storyboards are not silently rewritten. Older projects with repeated scenes should be rebuilt from the original ebook. +- Upload scratch files now use unique temporary names and are reliably cleaned up. + +### Generation, assembly, and resume reliability + +- Fixed a browser event-stream handler that could silently swallow server-side generation errors. +- Failed image generation no longer leaves a fake image path in browser state. +- Regeneration failures are displayed on the affected storyboard card. +- Final assembly is blocked until every required storyboard image exists, preventing partial or out-of-sync videos. +- Completed scene images are saved immediately and skipped when a session resumes. +- Recent-session video status now checks that the MP4 still exists on disk. +- Startup no longer terminates an unrelated program merely because port 5001 is occupied. +- Project, image, and download paths are validated before filesystem access. +- Storyboard and resume text inserted into HTML is escaped. +- Manual prompts and motion values are validated and clamped server-side. +- System-monitor GPU polling is cached briefly to reduce repeated `nvidia-smi` launches. +- Consolidated Windows priority and power behavior into the desktop runtime. + +### Packaging and repository hygiene + +- Added `BUILD.md` as the source of truth for every Windows BAT file. +- The release ZIP includes the BAT files for convenience, but GitHub ignores `*.bat`, `*.cmd`, and `*.lnk`. +- Added `.gitignore` coverage for virtual environments, model caches, runtime state, logs, generated output, uploads, shortcuts, and diagnostics. +- Added Windows CI for Python compilation and unit/regression tests. +- Added tests covering release hygiene, model configuration, installer/runtime download separation, EPUB duplication, repeated-scene avoidance, validation, and timeline behavior. + +### Validation completed before publication + +- Python source compilation completed successfully. +- JavaScript syntax validation completed successfully. +- Unit and regression tests completed successfully. +- GitHub Actions release-branch validation completed successfully. +- Every packaged BAT file exactly matches its corresponding code block in `BUILD.md`. +- Release-hygiene checks confirm the retired provider name is absent and BAT files remain ignored. +- ZIP integrity and packaged file checks completed successfully. + +### Upgrade notes + +- Install into a fresh SeeStory folder and run `install_all.bat`. +- The required local model is installed and GPU-tested before SeeStory is considered ready. +- Choose Yes during installation to install the optional Photorealistic model. +- Rebuild storyboards created with older builds when they contain repeated text or repeated prompts. +- Model weights are not included in GitHub or the release ZIP and remain governed by their upstream licenses. diff --git a/README.md b/README.md index 3b3e7b5..41e2fc6 100644 --- a/README.md +++ b/README.md @@ -2,279 +2,99 @@ **Bring your narrated audiobook to life.** -SeeStory turns a finished [Parroty](https://github.com/pgotta/Parroty) audiobook -into a watch‑along illustrated MP4. It reads the *same* ebook with the *same* -parser Parroty uses, lays the chapters onto Parroty's narration MP3, breaks each -chapter into timed "shots," draws a picture for each one, gives every still a -slow Ken Burns drift, and stitches the whole thing into one chaptered MP4 — -perfectly in sync, because each picture is timed to its slice of the narration. - -It runs entirely on your machine at **http://127.0.0.1:5001**, so it sits happily -beside Parroty (which uses port 5000). - -![SeeStory header and storyboard](screenshots/header.png) - ---- +SeeStory turns a finished [Parroty](https://github.com/pgotta/Parroty) audiobook into a synchronized illustrated MP4. It reads the same ebook, aligns it to Parroty's narration timestamps, builds a timed storyboard, generates each scene locally on the GPU, adds slow Ken Burns motion, and assembles the result with audio, subtitles, and chapter bookmarks. -## Table of Contents - -- [Requirements & disk space](#requirements--disk-space) -- [Quick start (Windows)](#quick-start-windows) -- [The image sources](#the-image-sources) -- [Choosing the look](#choosing-the-look) -- [Using it](#using-it) -- [Motion controls](#motion-controls) -- [Cover, subtitles, resume](#cover-subtitles-resume) -- [What you get](#what-you-get) -- [Optional: enable Copilot](#optional-enable-copilot) -- [Troubleshooting](#troubleshooting) -- [The launchers (.bat files)](#the-launchers-bat-files) -- [Credits](#credits) +SeeStory runs locally at **http://127.0.0.1:5001**. ---- +## Windows quick start -## Requirements & disk space +1. Extract the entire SeeStory folder to a normal writable location. +2. Double-click **`install_all.bat`**. It downloads, warm-loads, and performs a real GPU test with the required illustration model before the app is considered installed. It then asks whether to install the optional Photorealistic model too. +3. Start SeeStory from the desktop shortcut, or use **`run.bat`**. +4. Closing the dedicated SeeStory window stops the local server, just like **`stop.bat`**. -> **FYI — read before installing.** The local image models are large. Make sure -> you have the room and a supported GPU. +Have these Parroty outputs ready: -**Tested on:** Windows 11, **NVIDIA RTX 5060 Laptop GPU (8 GB VRAM)**, -PyTorch + CUDA, Python 3.12. It's a solo‑developer project built and run on that -exact setup — other configurations should work but haven't been exercised. +- The same ebook used to create the audiobook. +- Parroty's combined MP3. +- Parroty's `youtube-chapters-*.txt`, or equivalent timestamp lines. -| Need | Detail | -|------|--------| -| OS | Windows 10/11 (the `.bat` launchers are Windows‑only) | -| GPU | NVIDIA with CUDA. **8 GB VRAM is enough** thanks to fp16 + attention slicing + VAE tiling + CPU offload. CPU‑only works but is *very* slow. | -| ffmpeg | Required — it does all the video work. `winget install ffmpeg` | -| Python | 3.10+ (3.12 recommended) | +## Local image generation -**Disk space — budget ~20–25 GB free:** +There is one image-generation path and it runs locally. The web page does not expose provider/source controls. -| Item | Approx. size | When | -|------|-------------|------| -| Python venv (PyTorch + CUDA + diffusers) | ~6–8 GB | at setup | -| Stable Diffusion turbo model | ~7 GB | first time you generate with SD | -| Photorealistic model (RealVisXL Lightning) | ~7 GB | first time you pick the **Photorealistic** style | -| Copilot's Chromium (optional) | ~0.5 GB | only if you enable Copilot | -| Your generated images / clips / MP4 | varies (a long book can be several GB) | as you build | - -Models are downloaded once and cached in your Hugging Face cache -(`%USERPROFILE%\.cache\huggingface`), so they're only fetched the first time. -You can skip the Stable Diffusion download entirely with `setup.bat nosd` and -still run the whole pipeline on placeholder frames (and Copilot, if enabled). - ---- - -## Quick start (Windows) +- **DreamShaper XL Lightning** is selected automatically for Cinematic, Storybook, Noir, Oil, Ink, and other illustrated looks. +- **RealVisXL V5 Lightning** is selected automatically for the Photorealistic style. +- Prompts include stronger scene-coherence and human-anatomy guidance, plus negative prompting for extra limbs/fingers, duplicate people, malformed faces, and accidental text. +- `install_all.bat` downloads the required DreamShaper model (roughly 7 GB), warm-loads it, and creates a real smoke-test image on CUDA. The installer optionally offers the separate Photorealistic model. Windows security software may scan new weight files during installation. +- The running Flask app is offline-only for model access. It never starts a multi-gigabyte download; a missing/incomplete model produces a clear instruction to rerun the installer. +- The weights are not bundled in the SeeStory ZIP/repository and remain subject to each model's upstream license. +- On lower-VRAM NVIDIA GPUs, SeeStory automatically uses model CPU offload and can retry at smaller resolutions if a generation runs out of VRAM. -1. **`setup.bat`** — builds the virtual environment, installs the app, then - installs the local Stable Diffusion backend (PyTorch/CUDA + diffusers). To - skip that big download for now, run `setup.bat nosd`. -2. **Install ffmpeg:** `winget install ffmpeg`, then restart the terminal. -3. **`run.bat`** — Chrome opens at http://127.0.0.1:5001. Keep this window open - while generating. +No diffusion model can guarantee perfect hands or anatomy every time, so the storyboard keeps **Regenerate** manual and visible for individual scenes. -Have these three ready (all come from Parroty): the **same ebook** you fed -Parroty, Parroty's combined **MP3**, and Parroty's **`youtube-chapters‑*.txt`** -(or paste the `MM:SS Title` lines). Those timestamps are how the pictures line -up with the voice. +## Windows desktop behavior -> Copilot is **optional** and off by default — see -> [Optional: enable Copilot](#optional-enable-copilot). SeeStory is fully -> functional without it. +- Opens maximized in a dedicated Chrome or Edge app window. +- Uses `pythonw.exe`, so no PowerShell/Python console remains visible. +- Generates the desktop PNG/ICO assets from `build_icons.py` during installation, so a repository clone retains the same high-resolution icon without tracking generated binaries. +- Reuses one isolated SeeStory browser profile instead of creating a fresh profile every launch. This avoids the burst of browser/profile windows seen during startup while keeping SeeStory separate from the normal browser profile. +- Uses a single-instance launcher guard so an accidental double-click cannot start competing desktop sessions. +- Uses a lightweight startup health check, and Windows subprocesses such as ffmpeg/ffprobe run with hidden console flags, so helper processes do not flash extra windows. +- Runs the SeeStory server at Windows **HIGH** process priority. +- Disables Windows execution-speed throttling and browser background throttling. +- Prevents sleep while a long SeeStory session is active. +- Displays CPU, system RAM, GPU usage, VRAM, and GPU temperature in the lower-left corner. +- Uses browser-process tracking plus a heartbeat fallback for reliable shutdown. +- Writes launcher/server/lifecycle logs under `logs/`. ---- +If closing the app window ever leaves SeeStory running, run **`shutdown_diagnostic.bat`** before manually ending the process. It creates a timestamped report with the listener, PID, session state, GPU state, and recent logs. -## The image sources +## Requirements -You choose per book, and can override any single shot by hand: +- Windows 10 or Windows 11. +- Python 3.12 required by the Windows installer. +- NVIDIA CUDA GPU required for local image generation. The installer uses the CUDA 12.8 PyTorch path used by RTX 50-series cards and stops if real CUDA execution cannot be proven. +- ffmpeg for motion clips and final video assembly. +- Enough free disk space for the required model cache, optional Photorealistic cache, and generated images/video. The installer shows and verifies each model before completion. -- **Stable Diffusion (local)** — the every‑page workhorse. Free, private, runs on - your own GPU, no account, no limits. The default for most shots. -- **Copilot (optional, premium)** — drives Microsoft Copilot's own image - generation via the optional - [Windows‑Copilot‑API](https://github.com/sums001/Windows-Copilot-API). - Gorgeous, but it's a personal bridge processed one request at a time, so - SeeStory **saves it for the standout moments**, spaces the calls apart, and - enforces a **hard cap per book**. Entirely optional. -- **Placeholder** — a captioned frame that needs nothing installed. The fallback, - and it lets you test the whole pipeline (timing → motion → final video) before - downloading any models. +The installer does not permit a silent CPU fallback. -**Modes:** *Both* (SD everywhere, Copilot promoted to the most exciting moments), -*Stable Diffusion only* (no credentials at all), or *Copilot only* (capped and -spaced). If a backend isn't available, SeeStory quietly falls back rather than -failing. +## Workflow ---- +1. Add the ebook, narration audio, and chapter timestamps. +2. Pick the art style and how often the picture should change. +3. Build and review the storyboard; edit prompts or motion where useful. +4. Generate the images. Completed shots are saved immediately and are skipped when a session resumes. +5. Stitch the complete storyboard with narration, subtitles, and chapters into the MP4. -## Choosing the look +Video assembly is blocked if any storyboard image is missing, preventing a partially generated book from being stitched out of sync. -![Setup screen with a photorealistic sample](screenshots/setup-photoreal.png) +## Output -**Art style** (dropdown): +Each project folder under `output/` can contain: -- **Photorealistic** — looks like a real photo, not a drawing. Loads a dedicated - photo‑quality model (downloads ~7 GB the first time, then cached). Best for - realistic faces and scenery. -- **Cinematic / Storybook / Noir / Oil / Ink** — illustration looks, drawn with - the fast local model. +- Final chaptered SeeStory MP4. +- YouTube chapter timestamp file. +- Google Drive chapter page. +- Optional `.srt` subtitle file. +- Individual scene images and motion clips. +- `project.json` for resume/restoration. -![Storybook watercolor sample](screenshots/style-storybook.png) +## Repository and Windows launcher rules -**Custom style** (optional) overrides the preset — but put **visual style words -only** (medium, palette, lighting), never the plot, genre, author, or the word -"book." Genre/author words get painted into the image as garbled text. +Windows `.bat`, `.cmd`, and `.lnk` files are intentionally excluded from git. The test/release ZIP includes the BAT launchers, while **[BUILD.md](BUILD.md)** contains their exact contents so they can always be recreated from the repository. -**Image cleanup** controls how hard Stable Diffusion works to avoid stray text, -watermarks and distorted/double faces: +The repository also excludes the virtual environment, runtime state, logs, generated output, uploads, and local model caches. -- **Cleaner** (recommended) — removes most accidental text and duplicate faces. -- **Classic turbo** — fastest, original look, but artifacts can slip in. -- **Strong** — strongest cleanup and prompt adherence; slowest, can look a touch - oversaturated. Best when faces double up. +See **[CHANGELOG.md](CHANGELOG.md)** for the complete release history, fixes, and validation summary. -Use **Generate a sample image** to preview the exact style, model and cleanup -setting on one random page before committing to a full build. - ---- - -## Using it - -![The storyboard](screenshots/storyboard.png) - -1. **Bring in the book** — the same ebook, Parroty's MP3, and the chapter - timestamps. -2. **Choose the look** — mode, art style, words‑per‑page (how often the picture - changes), and the Copilot interval + hard cap. -3. **The storyboard** — every card is one shot, timed to its narration slice. - Edit its prompt, switch its source (SD / Copilot / Plain), or open **motion** - to tune it. ★ highlight shots are the ones routed to Copilot. -4. **Generate** — watch each shot fill in. Copilot shots pause a few seconds - between calls on purpose — that's the throttle. -5. **Stitch** — each still becomes a Ken Burns clip exactly as long as its - narration slice; they're joined and the MP3 is laid underneath with the same - chapter bookmarks Parroty made. Download the MP4. - -**One click does it all:** **✨ Generate all & build video →** runs generation and -stitching back to back and drops you at the finished MP4, download links, a build -log, and a scene‑breakdown table. Re‑running generation **skips images already -done**, so it always picks up where it left off. - -**Regenerate** and **Delete** are always yours to press — nothing ever redraws or -removes a shot on its own. (Regenerating a shot also refreshes its prompt with -the latest clean‑up logic, unless you've hand‑edited that prompt.) - ---- - -## Motion controls - -Per shot (or applied to all at once), with live sliders: - -- **zoom** — none / in / out -- **pan** — none / left / right / up / down -- **intensity** — how far the image travels -- **speed** — easing: quick start vs. slow build -- **fade in / fade out** — seconds up from / down to black -- **opacity** — gently dims the clip toward black for a "lights‑down" look - -This is motion on a still (ffmpeg `zoompan`), not generative video — the slow -drift‑and‑fade you want, fast and free enough to run on a whole book. Six presets -are included (gentle drift, slow reveal, pan left/right, dramatic push, still). -Hit **▶ Preview motion** to watch a short clip before you build. - ---- - -## Cover, subtitles, resume - -**Book cover (optional):** drop a cover image alongside the ebook and it plays as -a short title card at the very start, fading into the first illustration. Chapter -bookmarks (and the YouTube/Drive chapter files) shift to match, with a 0:00 cover -entry so YouTube still accepts them. - -**Subtitles (optional):** *off*, *soft* (a track viewers can toggle), or -*burned‑in*. For exact timing, drop in the `.srt` Parroty exports (its timings are -ground truth). Otherwise SeeStory auto‑generates approximate subtitles from the -ebook text. - -**Resume / restore:** every session is saved as it goes. If SeeStory is closed, -crashes, or runs out of memory, reopen it and a **Resume a recent session** panel -appears — pick the project and continue. - ---- - -## What you get - -In `output/-/`: - -- `seestory-.mp4` — the final video, with embedded chapter bookmarks. -- `youtube-chapters-.txt` — paste into a YouTube description for auto chapters. -- `drive-chapters-.html` — a clickable chapter index for Google Drive playback. -- `subtitles-.srt` — if subtitles were enabled. -- `images/` and `clips/` — the individual stills and motion clips. -- `project.json` — the whole storyboard, so a project survives a restart. - ---- - -## Optional: enable Copilot - -Copilot is an **optional** premium image source — SeeStory works fully without -it. The Windows‑Copilot‑API library it uses is **not included in this repo**; add -it only if you want Copilot. See **[BUILD.md](BUILD.md#optional-the-copilot-library)** -for the one‑time setup (clone the library into a `Windows-Copilot-API` folder, -then run the Copilot launchers). - -Once the library is in place: - -1. Run **`setup_copilot.bat`** — installs its dependencies and the Playwright - Chromium used for sign‑in (or `setup.bat` does this automatically when the - `Windows-Copilot-API` folder is present). -2. Double‑click **`login_copilot.bat`** — a Google Chrome window opens for a - one‑time Microsoft sign‑in and closes itself when done. -3. Restart SeeStory. The **copilot** badge turns green. Click **Test Copilot** in - the header to fire one real call and confirm it works; if it doesn't, the - **Copilot not working?** link explains exactly what to do. - -Copilot has content filters and may decline some prompts — those shots fall back -to Stable Diffusion automatically. Advanced: point it elsewhere with -`SEESTORY_COPILOT_PATH`. If Microsoft changes their protocol and you get an -"invalid‑event" error, update the `Windows-Copilot-API\copilot` folder from -[the repo](https://github.com/sums001/Windows-Copilot-API). - ---- - -## Troubleshooting - -- **Keep the console window in the foreground while generating.** Background - throttling on laptops can starve the GPU; `run.bat` raises process priority and - disables console QuickEdit to help. -- **ffmpeg missing?** Nothing renders. Install it (above) and restart. -- **Out of VRAM on 8 GB?** SeeStory auto‑retries the image at a smaller size and, - failing that, drops a placeholder for that one shot rather than crashing the - run — which, with resume, means a long book finishes even on a tight GPU. -- **Garbled text in images?** Clear the custom‑style box (or use visual words - only) and set image cleanup to **Strong**. -- **Faces doubling up?** Set image cleanup to **Strong**, or use the - **Photorealistic** style (its model handles faces far better). -- **Environment overrides:** `SEESTORY_SD_MODEL` (base model), - `SEESTORY_SD_PHOTOREAL_MODEL` (photo model), `SEESTORY_SD_GUIDANCE` - (force a guidance value), `SEESTORY_SD_NEGATIVE` (negative prompt). - ---- +## Credits -## The launchers (.bat files) +Made to run beside [Parroty](https://github.com/pgotta/Parroty). SeeStory's book parsing, scene planning, local image generation, and video build all run on the local computer. -The Windows launchers (`setup.bat`, `run.bat`, etc.) are **not tracked in git** — -they're generated/kept locally. If you cloned this repo and don't have them, see -**[BUILD.md](BUILD.md)** for the full contents and how to recreate each one. ---- +## Storyboard text and repeated scenes -## Credits +SeeStory reads EPUB text in DOM/spine order and avoids counting wrapper HTML containers twice. Shot text is non-overlapping, and the prompt director avoids reusing the same recent scene when another concrete sentence is available. If a storyboard was created by an older build and already contains repeated prompts, rebuild that storyboard from the original ebook; saved old project JSON is intentionally not rewritten behind your back. -Made to run beside [Parroty](https://github.com/pgotta/Parroty). The optional -Copilot backend uses the optional -[Windows‑Copilot‑API](https://github.com/sums001/Windows-Copilot-API) by sums001. -Nothing leaves your computer except, if you choose the Copilot backend, the -prompts you send to it. diff --git a/app/assembler.py b/app/assembler.py index 2164b3e..b1aa1fa 100644 --- a/app/assembler.py +++ b/app/assembler.py @@ -24,7 +24,7 @@ def _no_window_kwargs(): def ensure_ffmpeg() -> bool: try: - subprocess.run(["ffmpeg", "-version"], capture_output=True, check=True) + subprocess.run(["ffmpeg", "-version"], capture_output=True, check=True, **_no_window_kwargs()) return True except (subprocess.CalledProcessError, FileNotFoundError): return False @@ -69,8 +69,12 @@ def assemble_video(clip_paths: list, audio_path: str, out_path: str, work = os.path.dirname(out_path) list_file = os.path.join(work, "_concat.txt") with open(list_file, "w", encoding="utf-8") as f: - for p in clip_paths: - f.write(f"file '{os.path.abspath(p)}'\n") + for clip_path in clip_paths: + # ffconcat accepts forward slashes on Windows. Escape apostrophes so + # projects also work from paths such as C:/Users/O'Brien/.... + abs_path = os.path.abspath(clip_path).replace("\\", "/") + quoted = abs_path.replace("'", "'\\''") + f.write(f"file '{quoted}'\n") meta_file = None if markers and total_ms: diff --git a/app/copilot_login.py b/app/copilot_login.py deleted file mode 100644 index 894ba4a..0000000 --- a/app/copilot_login.py +++ /dev/null @@ -1,90 +0,0 @@ -""" -SeeStory Copilot sign-in launcher. - -Runs the Windows-Copilot-API interactive login, but with two SeeStory-specific -fixes over a bare ``python -m copilot login``: - -1. It launches your system **Google Chrome** (Playwright ``channel="chrome"``) - instead of Playwright's bundled Chromium, falling back to the bundled engine - only if Chrome isn't usable on this machine. -2. It runs from the SeeStory folder, so the saved session lands in - ``SeeStory\\session\\`` — exactly the working directory the app uses at - runtime, so the sign-in is actually found when generating images. - -Invoked by setup_copilot.bat and login_copilot.bat as: python -m app.copilot_login -""" - -import os -import sys - - -def _find_repo(): - """Locate the Windows-Copilot-API checkout next to (or inside) SeeStory.""" - env = os.environ.get("SEESTORY_COPILOT_PATH") - if env and os.path.isdir(os.path.join(env, "copilot")): - return os.path.abspath(env) - here = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) # SeeStory root - for cand in ("Windows-Copilot-API", os.path.join("..", "Windows-Copilot-API")): - full = os.path.abspath(os.path.join(here, cand)) - if os.path.isdir(os.path.join(full, "copilot")): - return full - return None - - -def _prefer_chrome(): - """Make Playwright's chromium launches use the system Google Chrome. - - Wraps BrowserType.launch_persistent_context so it adds channel="chrome". - If Chrome isn't installed/usable, it retries without the channel and you - get the bundled Chromium — so this never blocks sign-in. - """ - try: - from playwright.sync_api import BrowserType - except Exception: - return # playwright not importable; nothing to patch - - original = BrowserType.launch_persistent_context - - def patched(self, user_data_dir, **kwargs): - wants_default_chromium = ( - getattr(self, "name", "") == "chromium" - and "channel" not in kwargs - and "executable_path" not in kwargs - ) - if wants_default_chromium: - try: - return original(self, user_data_dir, channel="chrome", **kwargs) - except Exception as exc: - print(f" (Google Chrome not usable here [{exc.__class__.__name__}] " - f"- falling back to the bundled browser)") - return original(self, user_data_dir, **kwargs) - - BrowserType.launch_persistent_context = patched - - -def main(): - repo = _find_repo() - if not repo: - print("Windows-Copilot-API was not found. Run setup_copilot.bat first.") - return 1 - if repo not in sys.path: - sys.path.insert(0, repo) - - _prefer_chrome() - - try: - from copilot.browser import BrowserCopilot - except Exception as exc: - print(f"Could not import the Copilot library: {exc}") - print("Try running setup_copilot.bat again to (re)install its requirements.") - return 1 - - print("Opening Google Chrome for Microsoft / Copilot sign-in...\n") - BrowserCopilot(headless=False).login() # writes ./session relative to cwd - print("\nSigned in. Session saved to: " + os.path.abspath("session")) - print('The "copilot" badge in SeeStory should be green after a restart.') - return 0 - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/app/desktop_runtime.py b/app/desktop_runtime.py new file mode 100644 index 0000000..c8e32e2 --- /dev/null +++ b/app/desktop_runtime.py @@ -0,0 +1,310 @@ +"""Windows desktop runtime support for SeeStory. + +This module is intentionally separate from the audiobook/video pipeline. It only +handles desktop lifecycle, diagnostics, Windows performance settings, and the +small system-monitor endpoint used by the web UI. +""" +from __future__ import annotations + +import ctypes +import json +import os +import signal +import subprocess +import threading +import time +from pathlib import Path +from typing import Any + +from flask import jsonify, request + +_BASE = Path.cwd() +_PORT = 5001 +_DESKTOP = os.environ.get("SEESTORY_DESKTOP_SESSION") == "1" +_TOKEN = os.environ.get("SEESTORY_SESSION_TOKEN", "") +_STARTED = time.time() +_LAST_HEARTBEAT = 0.0 +_HEARTBEAT_SEEN = False +_SHUTTING_DOWN = False +_LOCK = threading.Lock() +_CPU_LAST: tuple[int, int] | None = None +_GPU_LAST = 0.0 +_GPU_CACHE: dict[str, Any] = {"available": False} + + +def _runtime_dir() -> Path: + path = Path(os.environ.get("SEESTORY_RUNTIME_DIR", str(_BASE / "runtime"))) + path.mkdir(parents=True, exist_ok=True) + return path + + +def _logs_dir() -> Path: + path = _BASE / "logs" + path.mkdir(parents=True, exist_ok=True) + return path + + +def _event(name: str, **details: Any) -> None: + payload = { + "time": time.strftime("%Y-%m-%d %H:%M:%S"), + "event": name, + "pid": os.getpid(), + **details, + } + try: + with (_logs_dir() / "desktop_runtime.log").open("a", encoding="utf-8") as fh: + fh.write(json.dumps(payload, ensure_ascii=False) + "\n") + except Exception: + pass + + +def _write_session(state: str = "running", **extra: Any) -> None: + payload = { + "pid": os.getpid(), + "port": _PORT, + "token": _TOKEN, + "desktop": _DESKTOP, + "state": state, + "started": _STARTED, + "updated": time.time(), + **extra, + } + try: + path = _runtime_dir() / "seestory-session.json" + tmp = path.with_suffix(".tmp") + tmp.write_text(json.dumps(payload, indent=2), encoding="utf-8") + tmp.replace(path) + except Exception: + pass + + +def _is_local_request() -> bool: + return request.remote_addr in {"127.0.0.1", "::1", None} + + +def _token_matches() -> bool: + supplied = request.headers.get("X-SeeStory-Token", "") + if not supplied and request.is_json: + supplied = (request.get_json(silent=True) or {}).get("token", "") + return bool(_TOKEN) and supplied == _TOKEN + + +def request_shutdown(reason: str) -> bool: + global _SHUTTING_DOWN + with _LOCK: + if _SHUTTING_DOWN: + return False + _SHUTTING_DOWN = True + _event("shutdown_requested", reason=reason) + _write_session("stopping", reason=reason) + + def terminate() -> None: + time.sleep(0.45) + try: + # This mirrors stop.bat semantics: end the local Flask process and + # all of its worker threads immediately after state is persisted. + os._exit(0) + except Exception: + os.kill(os.getpid(), signal.SIGTERM) + + threading.Thread(target=terminate, name="SeeStoryShutdown", daemon=True).start() + return True + + +def install(app, *, base_dir: str, port: int) -> None: + """Register desktop-only support routes without changing app behavior.""" + global _BASE, _PORT + _BASE = Path(base_dir) + _PORT = int(port) + + @app.post("/api/desktop/heartbeat") + def desktop_heartbeat(): + global _LAST_HEARTBEAT, _HEARTBEAT_SEEN + if not _is_local_request(): + return jsonify({"ok": False}), 403 + _LAST_HEARTBEAT = time.monotonic() + _HEARTBEAT_SEEN = True + return jsonify({"ok": True, "desktop": _DESKTOP}) + + @app.post("/api/desktop/shutdown") + def desktop_shutdown(): + if not _is_local_request() or not _token_matches(): + return jsonify({"ok": False}), 403 + body = request.get_json(silent=True) or {} + reason = str(body.get("reason") or "desktop_window_closed")[:80] + changed = request_shutdown(reason) + return jsonify({"ok": True, "already_stopping": not changed}) + + @app.get("/api/system") + def desktop_system(): + if not _is_local_request(): + return jsonify({"ok": False}), 403 + return jsonify(system_snapshot()) + + +def start() -> None: + """Apply Windows performance safeguards and start lifecycle diagnostics.""" + apply_windows_performance_mode() + snap = system_snapshot() + _event("startup", desktop=_DESKTOP, port=_PORT, system=snap) + _write_session("running", system=snap) + if _DESKTOP: + threading.Thread(target=_watchdog, name="SeeStoryHeartbeat", daemon=True).start() + + +def _watchdog() -> None: + # The tracked browser process normally requests shutdown immediately. This + # heartbeat is a second line of defense for browser crashes and odd Chrome + # process hand-offs. It intentionally waits for the first successful page + # heartbeat, so slow startup or a missing browser does not kill the server. + while not _SHUTTING_DOWN: + time.sleep(1.0) + if _HEARTBEAT_SEEN and time.monotonic() - _LAST_HEARTBEAT > 12.0: + _event("heartbeat_timeout", seconds=round(time.monotonic() - _LAST_HEARTBEAT, 1)) + request_shutdown("browser_heartbeat_lost") + return + + +def apply_windows_performance_mode() -> None: + if os.name != "nt": + return + try: + kernel32 = ctypes.windll.kernel32 + handle = kernel32.GetCurrentProcess() + # HIGH_PRIORITY_CLASS. This is the same intent as the prior launcher but + # no console window needs to remain focused for it to stay active. + kernel32.SetPriorityClass(handle, 0x00000080) + + # Keep Windows from applying execution-speed power throttling when the + # app window is covered or unfocused. + class POWER_THROTTLING_STATE(ctypes.Structure): + _fields_ = [ + ("Version", ctypes.c_uint32), + ("ControlMask", ctypes.c_uint32), + ("StateMask", ctypes.c_uint32), + ] + + state = POWER_THROTTLING_STATE(1, 0x1, 0) + kernel32.SetProcessInformation( + handle, 4, ctypes.byref(state), ctypes.sizeof(state) + ) + + # Keep the machine awake while a long book is generating. Windows + # automatically clears this when the SeeStory process exits. + kernel32.SetThreadExecutionState(0x80000000 | 0x00000001) + except Exception as exc: + _event("performance_mode_warning", error=str(exc)) + + +def _windows_cpu_percent() -> float | None: + global _CPU_LAST + if os.name != "nt": + try: + load = os.getloadavg()[0] + return round(min(100.0, load * 100.0 / max(1, os.cpu_count() or 1)), 1) + except Exception: + return None + try: + class FILETIME(ctypes.Structure): + _fields_ = [("low", ctypes.c_uint32), ("high", ctypes.c_uint32)] + + idle, kernel, user = FILETIME(), FILETIME(), FILETIME() + ctypes.windll.kernel32.GetSystemTimes( + ctypes.byref(idle), ctypes.byref(kernel), ctypes.byref(user) + ) + + def val(ft: FILETIME) -> int: + return (ft.high << 32) | ft.low + + idle_now = val(idle) + total_now = val(kernel) + val(user) + if _CPU_LAST is None: + _CPU_LAST = (idle_now, total_now) + return 0.0 + idle_prev, total_prev = _CPU_LAST + _CPU_LAST = (idle_now, total_now) + total_delta = max(1, total_now - total_prev) + busy = total_delta - max(0, idle_now - idle_prev) + return round(max(0.0, min(100.0, 100.0 * busy / total_delta)), 1) + except Exception: + return None + + +def _windows_memory() -> tuple[float | None, float | None]: + if os.name != "nt": + return None, None + try: + class MEMORYSTATUSEX(ctypes.Structure): + _fields_ = [ + ("length", ctypes.c_uint32), + ("memory_load", ctypes.c_uint32), + ("total_phys", ctypes.c_uint64), + ("avail_phys", ctypes.c_uint64), + ("total_page", ctypes.c_uint64), + ("avail_page", ctypes.c_uint64), + ("total_virtual", ctypes.c_uint64), + ("avail_virtual", ctypes.c_uint64), + ("avail_extended_virtual", ctypes.c_uint64), + ] + + stat = MEMORYSTATUSEX() + stat.length = ctypes.sizeof(stat) + ctypes.windll.kernel32.GlobalMemoryStatusEx(ctypes.byref(stat)) + used_gb = (stat.total_phys - stat.avail_phys) / (1024 ** 3) + total_gb = stat.total_phys / (1024 ** 3) + return round(used_gb, 1), round(total_gb, 1) + except Exception: + return None, None + + + +def _gpu_snapshot() -> dict[str, Any]: + """Read GPU stats with a short cache so the UI does not spawn nvidia-smi + every time multiple browser requests land close together.""" + global _GPU_LAST, _GPU_CACHE + now = time.monotonic() + if now - _GPU_LAST < 1.5: + return dict(_GPU_CACHE) + + cmd = [ + "nvidia-smi", + "--query-gpu=utilization.gpu,memory.used,memory.total,temperature.gpu,name", + "--format=csv,noheader,nounits", + ] + try: + flags = subprocess.CREATE_NO_WINDOW if os.name == "nt" else 0 + out = subprocess.check_output( + cmd, text=True, stderr=subprocess.DEVNULL, timeout=2.0, + creationflags=flags, + ).strip().splitlines() + if not out: + snap = {"available": False} + else: + util, used, total, temp, name = [x.strip() for x in out[0].split(",", 4)] + snap = { + "available": True, + "util": float(util), + "memory_used_mb": float(used), + "memory_total_mb": float(total), + "temperature_c": float(temp), + "name": name, + } + except Exception: + snap = {"available": False} + _GPU_CACHE = snap + _GPU_LAST = now + return dict(snap) + + +def system_snapshot() -> dict[str, Any]: + ram_used, ram_total = _windows_memory() + return { + "cpu_percent": _windows_cpu_percent(), + "ram_used_gb": ram_used, + "ram_total_gb": ram_total, + "gpu": _gpu_snapshot(), + "pid": os.getpid(), + "uptime_seconds": round(time.time() - _STARTED), + "priority": "HIGH" if os.name == "nt" else "normal", + "desktop_session": _DESKTOP, + } diff --git a/app/director.py b/app/director.py index ec84e85..385ab19 100644 --- a/app/director.py +++ b/app/director.py @@ -1,26 +1,18 @@ +"""Turn each timed text shot into one concise, visually coherent image prompt. + +The director is deliberately deterministic and local. It finds a concrete +sentence, strips dialogue/genre words that make image models draw text, and adds +consistent art direction. Human scenes receive extra anatomy guidance because +hands, limbs and duplicated bodies are common diffusion failure modes. """ -The "scene director": decide WHAT to draw for each shot and WHO draws it. - -Everything here runs locally with no credentials. For each shot it: - 1. finds the most visually concrete moment in the page text, - 2. writes an image-generation prompt, seeded with a persistent style bible - so recurring people/places look the same every time they appear, - 3. scores how exciting/visual the moment is, and - 4. routes the shot to a backend — Stable Diffusion for the everyday pages, - Copilot (higher quality) for the standout moments, under a hard cap so a - personal Copilot session is never hammered. - -An optional Copilot text pass can rewrite the prompt for the highlighted -shots, but it is never required — the heuristic alone produces usable prompts. -""" +from __future__ import annotations import re +from difflib import SequenceMatcher from typing import List from .timeline import Shot - -# Words that signal something worth *seeing* — concrete, paintable nouns. _IMAGERY = { "storm", "rain", "lightning", "thunder", "wind", "wave", "ocean", "sea", "lighthouse", "mountain", "forest", "tree", "river", "fire", "flame", @@ -31,8 +23,6 @@ "ballroom", "throne", "blood", "shadow", "dawn", "dusk", "sunset", "sunrise", "rose", "flower", "mansion", "cottage", "harbor", "valley", } - -# Words that signal a charged / pivotal moment worth a *premium* visual. _TENSION = { "suddenly", "scream", "screamed", "blood", "death", "died", "killed", "fire", "burning", "explosion", "gun", "knife", "fell", "crash", "storm", @@ -40,22 +30,26 @@ "chase", "ran", "fled", "battle", "fight", "war", "kiss", "kissed", "wept", "tears", "darkness", "terror", "horror", "monster", "ghost", } +_HUMAN = { + "man", "woman", "boy", "girl", "child", "person", "people", "mother", + "father", "mom", "dad", "brother", "sister", "husband", "wife", "soldier", + "doctor", "teacher", "officer", "he", "she", "him", "her", "his", "hers", +} _STOP_SENTENCE = re.compile(r"(?<=[.!?])\s+") _QUOTE = re.compile(r"[\"“”‘’']") _DIALOGUE = re.compile(r"^\s*[\"“].*?[\"”]\s*$") -_SPEECH_SPAN = re.compile(r"[\"“][^\"”]*[\"”]") # a run of quoted speech +_SPEECH_SPAN = re.compile(r"[\"“][^\"”]*[\"”]") def _strip_dialogue(s: str) -> str: - """Remove quoted speech from a sentence, leaving the descriptive remainder.""" s = _SPEECH_SPAN.sub("", s or "") s = _QUOTE.sub("", s) s = re.sub(r"\s{2,}", " ", s) return s.strip(" ,;:—-") -def _words(text: str): +def _words(text: str) -> list[str]: return re.findall(r"[a-z']+", (text or "").lower()) @@ -67,66 +61,92 @@ def _tension_score(text: str) -> int: return sum(1 for w in _words(text) if w in _TENSION) -def pick_focus_sentence(text: str, max_chars: int = 240) -> str: - """The single most paintable sentence in the passage.""" +def _has_human(text: str) -> bool: + return any(w in _HUMAN for w in _words(text)) + + +def _focus_candidates(text: str, max_chars: int = 280) -> list[str]: + """Return paintable sentences from best to worst, without duplicates.""" sents = [s.strip() for s in _STOP_SENTENCE.split(text or "") if s.strip()] if not sents: - return "" - # Prefer concrete, descriptive prose. Dialogue and questions ("You trying to - # tell us what to do?") describe no scene and make the image model hallucinate, - # so they're pushed down hard. + return [] scored = [] - for s in sents: - score = _imagery_score(s) * 2 + _tension_score(s) - if _DIALOGUE.match(s): # a whole line of speech - score -= 4 - if "?" in s or "!" in s: # questions/exclamations: usually speech + for idx, sent in enumerate(sents): + score = _imagery_score(sent) * 2 + _tension_score(sent) + if _DIALOGUE.match(sent): + score -= 5 + if "?" in sent: + score -= 3 + if sent.lstrip()[:1] in '"“‘\'': score -= 2 - if s.lstrip()[:1] in '"“‘\'': # starts mid-dialogue - score -= 2 - scored.append((score, len(s), s)) - scored.sort(key=lambda t: (-t[0], abs(t[1] - 140))) - best = _strip_dialogue(scored[0][2]) - if not best: # whole sentence was quoted speech - best = _QUOTE.sub("", scored[0][2]) - return best[:max_chars].strip() + score -= abs(len(sent) - 150) / 180.0 + scored.append((score, idx, sent)) + scored.sort(key=lambda item: item[0], reverse=True) + + out: list[str] = [] + seen: set[str] = set() + for _score, idx, raw in scored: + best = _strip_dialogue(raw) or _QUOTE.sub("", raw) + low = _words(best) + pronoun_heavy = bool(low) and low[0] in {"he", "she", "they", "his", "her", "their"} + if pronoun_heavy and idx > 0: + context = _strip_dialogue(sents[idx - 1]) + if context and len(context) <= 140: + best = f"{context}. {best}" + best = best[:max_chars].strip() + key = re.sub(r"[^a-z0-9]+", " ", best.lower()).strip() + if best and key and key not in seen: + seen.add(key) + out.append(best) + return out + + +def pick_focus_sentence(text: str, max_chars: int = 280) -> str: + """Pick the most paintable non-dialogue sentence in a passage.""" + candidates = _focus_candidates(text, max_chars=max_chars) + return candidates[0] if candidates else "" + + +def _focus_key(text: str) -> str: + return re.sub(r"[^a-z0-9]+", " ", (text or "").lower()).strip() + + +def _too_similar_focus(candidate: str, previous: str) -> bool: + """Guard against near-identical consecutive storyboard scenes.""" + a, b = _focus_key(candidate), _focus_key(previous) + if not a or not b: + return False + if a == b: + return True + # Long scene sentences can differ by a couple of words yet still generate + # essentially the same picture. Keep this deliberately conservative. + return SequenceMatcher(None, a, b).ratio() >= 0.88 class StyleBible: - """The look of the book + consistent descriptions of recurring entities. - - `style` is appended to every prompt (the art direction). `entities` maps a - name -> a short fixed description; whenever a shot's text mentions that - name, the description is folded into the prompt so the character/place is - rendered consistently across the whole book. - """ - PRESETS = { - "photoreal": "photorealistic photograph, 85mm lens, natural light, " - "sharp focus, realistic skin texture, beautiful people " - "and scenery", - "cinematic": "cinematic painterly illustration, dramatic lighting, " - "rich depth of field, atmospheric, detailed", - "storybook": "warm storybook watercolor illustration, soft edges, " - "gentle light, hand-painted texture", - "noir": "moody film-noir illustration, high contrast, deep shadows, " - "rain-slicked, monochrome with a single warm accent", - "oil": "classical oil painting, visible brushwork, golden-hour light, " - "romantic realism", - "ink": "detailed pen-and-ink illustration with selective watercolor " - "washes, fine linework", + "photoreal": "photorealistic cinematic still, natural light, realistic skin texture, " + "anatomically correct proportions, coherent composition, sharp detail", + "cinematic": "cinematic painterly illustration, dramatic natural lighting, rich depth, " + "coherent composition, detailed", + "storybook": "warm storybook watercolor illustration, soft edges, gentle light, " + "hand-painted texture, coherent composition", + "noir": "moody film-noir illustration, high contrast, deep shadows, rain-slicked, " + "monochrome with one warm accent, coherent composition", + "oil": "classical oil painting, visible brushwork, golden-hour light, romantic realism, " + "coherent composition", + "ink": "detailed pen-and-ink illustration with selective watercolor washes, fine " + "linework, coherent composition", } - def __init__(self, style_key: str = "cinematic", - custom_style: str = "", entities: dict = None): + def __init__(self, style_key: str = "cinematic", custom_style: str = "", entities: dict | None = None): self.style_key = style_key self.custom_style = custom_style.strip() self.entities = entities or {} @property def style(self) -> str: - return self.custom_style or self.PRESETS.get(self.style_key, - self.PRESETS["cinematic"]) + return self.custom_style or self.PRESETS.get(self.style_key, self.PRESETS["cinematic"]) def entity_hints(self, text: str) -> str: low = (text or "").lower() @@ -134,16 +154,11 @@ def entity_hints(self, text: str) -> str: if name and name.lower() in low and desc] return "; ".join(hints) - def to_json(self): + def to_json(self) -> dict: return {"style_key": self.style_key, "custom_style": self.custom_style, "entities": self.entities} -# Words that make image models render literal text / book-cover titles, or that -# trip Copilot's content filter (so it declines and returns no image). We strip -# them from the art-direction style and the assembled prompt. This is why a -# "custom style" like "thriller, dan brown book" produced giant garbled titles -# and made Copilot bail — the model saw "book" and drew a cover. _BAD_PROMPT_TERMS = re.compile( r"\b(audio ?book|book|novel|ebook|e-book|paperback|hardcover|cover|" r"title|titled|chapter|page|text|words?|lettering|caption|subtitles?|" @@ -152,115 +167,76 @@ def to_json(self): r"science fiction|drama|comedy|crime|noir fiction|" r"adult[- ]?oriented|adults?|nsfw|explicit|erotic|porn(ographic)?|" r"gore|gory|graphic)\b", - re.I) + re.I, +) def _scrub(s: str) -> str: - """Drop text-inducing / filter-tripping tokens and tidy leftover commas.""" s = _BAD_PROMPT_TERMS.sub("", s or "") - s = re.sub(r"\s*,(?:\s*,)+", ", ", s) # collapse empty commas + s = re.sub(r"\s*,(?:\s*,)+", ", ", s) s = re.sub(r"\s{2,}", " ", s) return s.strip(" ,;") def _approx_tokens(s: str) -> int: - """Rough CLIP token count — enough to keep us under the 77-token limit.""" return int(len(s.split()) * 1.35) + s.count(",") def _trim_to_tokens(s: str, max_tokens: int) -> str: - """Keep whole words from the front until we'd exceed the budget.""" - out = [] - for w in s.split(): - out.append(w) + out: list[str] = [] + for word in s.split(): + out.append(word) if _approx_tokens(" ".join(out)) >= max_tokens: break return " ".join(out).rstrip(" ,;:—-") -def build_prompt(shot: Shot, bible: StyleBible) -> str: - """Compose a clean image prompt for a shot.""" - focus = pick_focus_sentence(shot.text) - focus = _QUOTE.sub("", focus) +def build_prompt(shot: Shot, bible: StyleBible, *, focus_override: str | None = None) -> str: + focus = _scrub(_QUOTE.sub("", focus_override if focus_override is not None + else pick_focus_sentence(shot.text))) focus = re.sub(r"\s+", " ", focus).strip().rstrip(".") if not focus: - focus = f"a quiet scene from {shot.chapter_title}" - hints = bible.entity_hints(shot.text) - style = _scrub(bible.style) - if not style: - # The custom style was all genre/non-visual words (e.g. "thriller") and - # scrubbed away — fall back to the chosen preset so every image still has - # real art direction instead of none. - style = bible.PRESETS.get(bible.style_key, bible.PRESETS["cinematic"]) - # CLIP only reads ~77 tokens. Reserve room for the style (and any entity - # hints), then trim the scene sentence to fit so nothing is silently cut — - # the scene stays first (most important) and the style always survives. - tail = ", ".join(p for p in (hints, style) if p) - budget = 70 - _approx_tokens(tail) - 1 - focus = _trim_to_tokens(focus, max(8, budget)) - parts = [focus] + ([hints] if hints else []) + [style] - return ", ".join(parts) - - -def score_highlight(shot: Shot) -> float: - """How much this moment deserves a premium (Copilot) render.""" - s = _tension_score(shot.text) * 2.0 + _imagery_score(shot.text) * 1.0 - if shot.is_chapter_start: - s += 1.5 - return round(s, 2) + focus = "a quiet atmospheric scene" + hints = _scrub(bible.entity_hints(shot.text)) + style = _scrub(bible.style) or bible.PRESETS.get( + bible.style_key, bible.PRESETS["cinematic"] + ) + anatomy = ( + "natural human anatomy, realistic hands, five fingers per hand, " + "two arms and two legs, no duplicated body parts" + if _has_human(shot.text) else "" + ) + coherence = "single coherent scene, one moment, physically plausible composition" -def direct(shots: List[Shot], bible: StyleBible) -> None: - """Fill prompt + highlight_score on every shot (in place).""" - for sh in shots: - sh.prompt = build_prompt(sh, bible) - sh.highlight_score = score_highlight(sh) + tail = ", ".join(part for part in (hints, anatomy, coherence, style) if part) + budget = 72 - _approx_tokens(tail) - 1 + focus = _trim_to_tokens(focus, max(10, budget)) + return ", ".join(part for part in (focus, hints, anatomy, coherence, style) if part) -def route_backends(shots: List[Shot], *, mode: str = "both", - sd_backend: str = "stablediffusion", - copilot_every_pages: int = 10, - copilot_cap: int = 30) -> dict: - """Assign .backend / .highlighted across all shots. +def direct(shots: List[Shot], bible: StyleBible) -> None: + """Write prompts while avoiding recycled scene choices within a chapter. - mode: 'sd_only' | 'copilot_only' | 'both' - Returns a small summary dict for the UI. + Normally each shot receives non-overlapping source text, so duplicate focus + sentences should be rare. EPUBs with repeated headers or unusual markup can + still expose the same sentence twice; in that case prefer the next-best + concrete sentence before allowing a repeated image concept. """ - total_pages = sum((s.page_end - s.page_start + 1) for s in shots) or 1 - - if mode == "copilot_only": - for s in shots: - s.backend, s.highlighted = "copilot", True - # still respect the cap: beyond the cap, fall back to SD/placeholder - for s in sorted(shots, key=lambda x: -x.highlight_score)[copilot_cap:]: - s.backend, s.highlighted = sd_backend, False - used = sum(1 for s in shots if s.backend == "copilot") - return {"copilot": used, "sd": len(shots) - used, "total": len(shots)} - - # default everything to the SD-class backend first - for s in shots: - s.backend, s.highlighted = sd_backend, False - - if mode == "sd_only": - return {"copilot": 0, "sd": len(shots), "total": len(shots)} - - # mode == 'both': promote the strongest moments to Copilot, but no more than - # roughly one per `copilot_every_pages` pages and never past the hard cap. - budget = min(copilot_cap, max(1, total_pages // max(1, copilot_every_pages))) - candidates = sorted(shots, key=lambda x: -x.highlight_score) - promoted, last_start = 0, {} - min_gap_ms = 60_000 # don't put two Copilot shots within a minute of audio - for s in candidates: - if promoted >= budget: - break - if s.highlight_score <= 0 and not s.is_chapter_start: - continue - # spacing: keep premium shots spread out across the runtime - too_close = any(abs(s.start_ms - t) < min_gap_ms for t in last_start.values()) - if too_close: - continue - s.backend, s.highlighted = "copilot", True - last_start[s.id] = s.start_ms - promoted += 1 - return {"copilot": promoted, "sd": len(shots) - promoted, - "total": len(shots), "budget": budget} + history: dict[int, list[str]] = {} + for shot in shots: + prior = history.setdefault(shot.chapter_index, []) + candidates = _focus_candidates(shot.text) + focus = "" + for candidate in candidates: + # Compare against the last few scenes in this chapter. Looking at a + # small window prevents obvious repetition without forcing distant, + # intentionally recurring motifs to become unrelated. + if not any(_too_similar_focus(candidate, old) for old in prior[-4:]): + focus = candidate + break + if not focus: + focus = candidates[0] if candidates else "" + shot.prompt = build_prompt(shot, bible, focus_override=focus) + if focus: + prior.append(focus) diff --git a/app/document_parser.py b/app/document_parser.py index a09845e..7a4ef62 100644 --- a/app/document_parser.py +++ b/app/document_parser.py @@ -20,6 +20,7 @@ import warnings from .epub_parser import Chapter, ParsedBook +from .html_text import semantic_blocks # ---- shared heading heuristics ------------------------------------------ @@ -205,13 +206,9 @@ def _parse_html(path: str, title: str) -> ParsedBook: bad.decompose() doc_title = (soup.title.get_text(strip=True) if soup.title else "") or title - blocks, flags = [], [] - for el in soup.find_all(["h1", "h2", "h3", "h4", "p", "div", "li"]): - text = el.get_text(" ", strip=True) - if not text: - continue - blocks.append(text) - flags.append(el.name in ("h1", "h2", "h3", "h4")) + pairs = semantic_blocks(soup) + blocks = [text for text, _is_heading in pairs] + flags = [is_heading for _text, is_heading in pairs] chapters = _chapters_from_blocks(blocks, flags) return ParsedBook(title=doc_title, author="", chapters=chapters) @@ -227,7 +224,10 @@ def _parse_doc(path: str, title: str) -> ParsedBook: # Try antiword via subprocess if installed. try: import subprocess - out = subprocess.run(["antiword", path], capture_output=True) + kwargs = {} + if os.name == "nt": + kwargs["creationflags"] = getattr(subprocess, "CREATE_NO_WINDOW", 0x08000000) + out = subprocess.run(["antiword", path], capture_output=True, **kwargs) if out.returncode == 0: text = out.stdout.decode("utf-8", errors="replace") except Exception: diff --git a/app/epub_parser.py b/app/epub_parser.py index 017896a..3eca009 100644 --- a/app/epub_parser.py +++ b/app/epub_parser.py @@ -15,6 +15,8 @@ from ebooklib import epub import ebooklib +from .html_text import clean_html_to_text as _clean_html_to_text + warnings.filterwarnings("ignore", category=XMLParsedAsHTMLWarning) @@ -37,33 +39,6 @@ class ParsedBook: _HEADING_TAGS = ("h1", "h2", "h3") -def _clean_html_to_text(html: str) -> str: - """Strip tags and collapse whitespace, keeping paragraph breaks.""" - soup = BeautifulSoup(html, "lxml") - - for bad in soup(["script", "style"]): - bad.decompose() - - # Turn block elements into newline-separated text so paragraphs survive. - for br in soup.find_all("br"): - br.replace_with("\n") - - blocks = [] - for el in soup.find_all(["p", "div", "h1", "h2", "h3", "h4", "h5", "h6", "li"]): - chunk = el.get_text(" ", strip=True) - if chunk: - blocks.append(chunk) - - if not blocks: - # Fallback: whole-document text. - blocks = [soup.get_text(" ", strip=True)] - - text = "\n\n".join(blocks) - text = re.sub(r"[ \t]+", " ", text) - text = re.sub(r"\n{3,}", "\n\n", text) - return text.strip() - - def _ordered_documents(book): """Yield the book's document items in spine (reading) order. diff --git a/app/html_text.py b/app/html_text.py new file mode 100644 index 0000000..a320d8d --- /dev/null +++ b/app/html_text.py @@ -0,0 +1,61 @@ +"""Shared HTML/EPUB text extraction helpers. + +The key rule is that every DOM text node must be read exactly once. Reading a +wrapper
and each nested

separately duplicates prose in many EPUBs. +""" +from __future__ import annotations + +import re +from typing import Iterable + +from bs4 import BeautifulSoup + +_BLOCK_TAGS = ( + "p", "div", "section", "article", "blockquote", "pre", + "h1", "h2", "h3", "h4", "h5", "h6", "li", +) +_SEMANTIC_TAGS = ("h1", "h2", "h3", "h4", "p", "li", "blockquote", "pre") +_HEADING_TAGS = {"h1", "h2", "h3", "h4"} + + +def make_soup(html: str) -> BeautifulSoup: + soup = BeautifulSoup(html or "", "lxml") + for bad in soup(["script", "style", "nav"]): + bad.decompose() + return soup + + +def clean_html_to_text(html: str) -> str: + """Return readable text in DOM order without duplicating nested containers.""" + soup = make_soup(html) + + for br in soup.find_all("br"): + br.replace_with("\n") + # Insert boundaries, then do a single text traversal. Nested blocks can add + # whitespace but cannot cause the underlying prose to be read twice. + for el in soup.find_all(_BLOCK_TAGS): + el.append("\n\n") + + text = soup.get_text(" ", strip=False).replace("\r", "") + text = re.sub(r"[ \t]*\n[ \t]*", "\n", text) + text = re.sub(r"[ \t]+", " ", text) + text = re.sub(r"\n{3,}", "\n\n", text) + return text.strip() + + +def semantic_blocks(html_or_soup) -> list[tuple[str, bool]]: + """Return (text, is_heading) blocks without wrapper-container duplication.""" + soup = html_or_soup if isinstance(html_or_soup, BeautifulSoup) else make_soup(html_or_soup) + blocks: list[tuple[str, bool]] = [] + for el in soup.find_all(list(_SEMANTIC_TAGS) + ["div"]): + # A div is useful only when it is a leaf-like text container. If it owns + # paragraphs/headings/list items, those semantic children carry the text. + if el.name == "div" and el.find(_SEMANTIC_TAGS): + continue + text = re.sub(r"\s+", " ", el.get_text(" ", strip=True)).strip() + if not text: + continue + if blocks and text == blocks[-1][0]: + continue + blocks.append((text, el.name in _HEADING_TAGS)) + return blocks diff --git a/app/imagegen/__init__.py b/app/imagegen/__init__.py index 86bfa6b..c08c51d 100644 --- a/app/imagegen/__init__.py +++ b/app/imagegen/__init__.py @@ -1,65 +1,35 @@ -"""Image-generation router: pick a backend per shot, fall back gracefully.""" +"""Local image-generation entry point. + +SeeStory intentionally has one image path: local diffusion generation. The UI +does not expose implementation choices, and generation failures are surfaced +instead of silently substituting different artwork. +""" import sys -from . import placeholder, stablediffusion, copilot_backend +from . import stablediffusion from ..director import _scrub -# fallback order if a backend is unavailable or errors on a given shot -_FALLBACK = { - "copilot": ["copilot", "stablediffusion", "placeholder"], - "stablediffusion": ["stablediffusion", "placeholder"], - "placeholder": ["placeholder"], -} - -_LABEL = {"copilot": "copilot", "stablediffusion": "stable diffusion", - "placeholder": "placeholder"} - def probe() -> dict: + """Small readiness snapshot used by the launcher and diagnostics.""" return { - "stablediffusion": stablediffusion.is_available(), + "ready": stablediffusion.is_available(), "cuda": stablediffusion.has_cuda(), - "copilot": copilot_backend.is_available(), - "copilot_signed_in": copilot_backend.is_signed_in(), - "copilot_remaining": copilot_backend.remaining(), + "model": stablediffusion.DEFAULT_MODEL, } -def generate_for(shot, out_path: str, *, sd_opts: dict = None) -> dict: - """Generate one shot's image. Returns {'backend': used, 'note': str}.""" +def generate_for(shot, out_path: str, *, sd_opts: dict | None = None) -> str: + """Generate one storyboard image locally and return its output path.""" sd_opts = sd_opts or {} - # Scrub text-inducing / filter-tripping tokens even from already-stored - # prompts (e.g. an old storyboard built with a "…dan brown book" style), - # so regenerating a shot benefits from the fix too. - prompt = _scrub(shot.prompt) or shot.prompt - chain = _FALLBACK.get(shot.backend, ["placeholder"]) - last_err = "" - for backend in chain: - try: - if backend == "copilot": - copilot_backend.generate(prompt, out_path) - elif backend == "stablediffusion": - stablediffusion.generate(prompt, out_path, **sd_opts) - else: - placeholder.generate(prompt, out_path, - label=_LABEL.get(shot.backend, "scene")) - note = "" if backend == shot.backend else \ - f"{shot.backend} unavailable → {backend}" - # carry the real reason so it's visible in the browser build log - if note and last_err: - note += f" — {last_err}" - return {"backend": backend, "note": note, "error": last_err} - except Exception as e: - last_err = str(e) - # Surface the reason on the console (→ seestory.log) so a backend - # silently falling back (e.g. Copilot) is never invisible again. - sys.stderr.write( - f"[seestory] {backend} failed for shot " - f"{getattr(shot, 'id', '?')}: {e}\n") - sys.stderr.flush() - continue - # placeholder is last resort and shouldn't fail, but just in case: - placeholder.generate(shot.prompt or "scene", out_path) - return {"backend": "placeholder", "note": "all backends failed", - "error": last_err} + prompt = _scrub(getattr(shot, "prompt", "")) or getattr(shot, "prompt", "") + try: + return stablediffusion.generate(prompt, out_path, **sd_opts) + except Exception as exc: + sys.stderr.write( + f"[seestory] local image generation failed for shot " + f"{getattr(shot, 'id', '?')}: {exc}\n" + ) + sys.stderr.flush() + raise diff --git a/app/imagegen/copilot_backend.py b/app/imagegen/copilot_backend.py deleted file mode 100644 index 71111ef..0000000 --- a/app/imagegen/copilot_backend.py +++ /dev/null @@ -1,255 +0,0 @@ -""" -Copilot backend (the premium, save-it-for-the-big-moments option). - -Wraps the Windows-Copilot-API (https://github.com/sums001/ -Windows-Copilot-API), which drives Microsoft Copilot's own image generation and -returns a hosted image URL. We download that URL to disk. - -Because that API is a *personal* bridge processed one request at a time, this -backend is wrapped in: - * a minimum spacing between calls (token bucket), and - * a hard per-run cap, -so an account is never hammered. It's optional: if the library isn't found or -you haven't logged in, the router falls back to Stable Diffusion / placeholder. - -Setup (one time): just run setup_copilot.bat, which downloads the repo into the -SeeStory folder, installs its deps, and signs you in via Google Chrome (saving the -session to SeeStory/session/). Re-sign-in later with login_copilot.bat. The folder -is auto-detected; override with SEESTORY_COPILOT_PATH if it lives elsewhere. -""" - -import os -import sys -import time -import threading -import urllib.request - -_LOCK = threading.Lock() -_CLIENT = None -_LAST_CALL = 0.0 -_COUNT = 0 - -MIN_SPACING_S = float(os.environ.get("SEESTORY_COPILOT_SPACING", "8")) -HARD_CAP = int(os.environ.get("SEESTORY_COPILOT_CAP", "30")) - - -def _find_copilot_path(): - p = os.environ.get("SEESTORY_COPILOT_PATH") - if p and os.path.isdir(p): - return p - here = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) - for cand in ("Windows-Copilot-API", os.path.join("..", "Windows-Copilot-API")): - full = os.path.abspath(os.path.join(here, cand)) - if os.path.isdir(os.path.join(full, "copilot")): - return full - return None - - -def is_available() -> bool: - try: - path = _find_copilot_path() - if path and path not in sys.path: - sys.path.insert(0, path) - import copilot # noqa: F401 - return True - except Exception: - return False - - -def _session_dir() -> str: - root = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) - return os.path.join(root, "session") - - -def is_signed_in() -> bool: - """True if a saved Copilot sign-in session exists on disk. (Whether it's - still *valid* can only be known for sure by making a call — but a saved - session is the right gate for the badge, and an expired one will surface a - clear error at generation time.)""" - sess = _session_dir() - if not os.path.isdir(sess): - return False - if os.path.exists(os.path.join(sess, "token.json")): - return True - try: - return len(os.listdir(sess)) > 0 - except OSError: - return False - - -def _client(): - global _CLIENT - if _CLIENT is None: - path = _find_copilot_path() - if path and path not in sys.path: - sys.path.insert(0, path) - from copilot import CopilotClient - _CLIENT = CopilotClient() - return _CLIENT - - -def remaining() -> int: - return max(0, HARD_CAP - _COUNT) - - -def enrich_prompt(prompt: str, source_text: str = "") -> str: - """Optional: let Copilot rewrite a richer image prompt. Best-effort.""" - try: - ask = ("Rewrite the following into one vivid, concrete image-generation " - "prompt (a single line, no preamble, no quotes, describe only " - "what is visible): " + prompt) - reply = _client().chat(ask) - line = (reply.text or "").strip().splitlines()[0].strip().strip('"') - return line or prompt - except Exception: - return prompt - - -_CONSEC_FAIL = 0 -_DISABLED_THIS_RUN = False - - -def reset_run_state(): - """Call at the start of a generation run: clears the per-run image cap and - the 'too many failures, stop trying' breaker.""" - global _COUNT, _CONSEC_FAIL, _DISABLED_THIS_RUN - _COUNT = 0 - _CONSEC_FAIL = 0 - _DISABLED_THIS_RUN = False - - -def test() -> tuple: - """Make ONE real end-to-end Copilot call to verify it works right now. - Returns (ok: bool, detail: str) — the detail is already user-friendly.""" - if not is_available(): - return False, ("Copilot library not found, or its dependencies aren't " - "installed. Run setup.bat, then login_copilot.bat.") - if not is_signed_in(): - return False, ("Not signed in. Run login_copilot.bat to sign in, then " - "restart SeeStory.") - reset_run_state() - import tempfile - tmp = os.path.join(tempfile.gettempdir(), "seestory_copilot_test.jpg") - try: - generate("a simple test illustration: a single blue circle on a white " - "background", tmp) - except Exception as e: - return False, str(e) # already translated by generate()/_explain() - ok = os.path.exists(tmp) and os.path.getsize(tmp) > 0 - try: - os.remove(tmp) - except OSError: - pass - if ok: - return True, "Copilot is working — a test image generated successfully." - return False, "Copilot returned no image (it may have declined the test prompt)." - - -def _explain(err: str) -> str: - """Turn a raw library error into something the user can act on.""" - e = (err or "").lower() - if "invalid-event" in e: - return ("Copilot rejected the request (invalid-event). The " - "Windows-Copilot-API library's chat handshake is out of step with " - "Microsoft's current Copilot protocol — update that library (or let " - "its author know). Using Stable Diffusion meanwhile.") - if "clearance" in e or "turnstile" in e or "cf_clearance" in e or "503" in e: - return ("Copilot needs fresh Cloudflare clearance — run login_copilot.bat " - "to refresh it, then retry.") - if "chat-service-unavailable" in e: - return ("Copilot's chat backend is geo-restricted / unavailable right now; " - "using Stable Diffusion.") - if "no active socket" in e or "websocket" in e or "connection" in e: - return ("Couldn't reach Copilot (connection issue); using Stable Diffusion. " - "If it persists, re-run login_copilot.bat.") - return "Copilot request failed: " + str(err) - - -def generate(prompt: str, out_path: str, **_) -> str: - global _LAST_CALL, _COUNT, _CONSEC_FAIL, _DISABLED_THIS_RUN - if not is_available(): - raise RuntimeError("Copilot backend unavailable (library not found / " - "not logged in). Falling back.") - if _DISABLED_THIS_RUN: - raise RuntimeError("Copilot turned off for the rest of this run after " - "repeated failures (see the earlier Copilot error). " - "Using Stable Diffusion.") - with _LOCK: - if _COUNT >= HARD_CAP: - raise RuntimeError(f"Copilot hard cap reached ({HARD_CAP}). " - "Using Stable Diffusion instead.") - wait = MIN_SPACING_S - (time.time() - _LAST_CALL) - if wait > 0: - time.sleep(wait) - - reply = None - err = None - ask = ("Generate a single illustration of the following scene. Output only " - "the image — do not reply with text, and put no text, words, " - "captions or watermark inside the picture. Scene: " + prompt) - for attempt in range(2): # one retry — invalid-event can be a transient race - try: - reply = _client().chat(ask) - err = None - break - except Exception as e: - err = e - if attempt == 0: - time.sleep(2.0) - _LAST_CALL = time.time() - - def _trip(): - global _CONSEC_FAIL, _DISABLED_THIS_RUN - _CONSEC_FAIL += 1 - if _CONSEC_FAIL >= 2: # two bad shots in a row -> stop trying for this run - _DISABLED_THIS_RUN = True - - if err is not None: - _trip() - raise RuntimeError(_explain(str(err))) - - if not getattr(reply, "images", None): - _trip() - attrs = ", ".join(a for a in dir(reply) if not a.startswith("_"))[:160] \ - if reply is not None else "None" - raise RuntimeError( - "Copilot returned no image (it may have declined this prompt, or " - f"the session isn't signed in). reply type={type(reply).__name__}; " - f"fields: {attrs}") - - url = reply.images[0].url - req = urllib.request.Request(url, headers={"User-Agent": "Mozilla/5.0"}) - with urllib.request.urlopen(req, timeout=60) as r: - data = r.read() - with open(out_path, "wb") as f: - f.write(data) - _COUNT += 1 - _CONSEC_FAIL = 0 # a success clears the breaker - return out_path - - -def reset_count(): - global _COUNT - _COUNT = 0 - - -def diagnose() -> str: - """A short human-readable status line for the console / build log, so it's - obvious WHY Copilot is or isn't usable before a run starts.""" - path = _find_copilot_path() - if not path: - return ("library folder not found — expected 'Windows-Copilot-API' next " - "to the SeeStory folder. Run setup_copilot.bat.") - bits = [f"found at {path}"] - if is_signed_in(): - bits.append("session present") - else: - bits.append("NO saved session — run login_copilot.bat to sign in") - try: - if path not in sys.path: - sys.path.insert(0, path) - import copilot # noqa: F401 - bits.append("library imports OK") - except Exception as e: - bits.append(f"import failed: {e}") - return "; ".join(bits) diff --git a/app/imagegen/placeholder.py b/app/imagegen/placeholder.py deleted file mode 100644 index 1afeec0..0000000 --- a/app/imagegen/placeholder.py +++ /dev/null @@ -1,58 +0,0 @@ -""" -Always-available image backend: a captioned gradient frame. - -It needs no GPU, no model download, and no login, so the full pipeline — -timing, Ken Burns motion, and final assembly — can be tested the moment the app -starts. Swap in Stable Diffusion or Copilot for real art when you're ready; the -storyboard, sync, and video build are identical either way. -""" - -import hashlib -import textwrap - -from PIL import Image, ImageDraw, ImageFont - - -def _hue_from(seed: str): - h = hashlib.sha256(seed.encode("utf-8")).digest() - # two muted, cinematic tones for a vertical gradient - top = (28 + h[0] % 40, 24 + h[1] % 40, 30 + h[2] % 50) - bot = (8 + h[3] % 24, 6 + h[4] % 20, 10 + h[5] % 26) - return top, bot - - -def _font(size): - for name in ("DejaVuSerif.ttf", "DejaVuSans.ttf"): - try: - return ImageFont.truetype(name, size) - except Exception: - continue - return ImageFont.load_default() - - -def generate(prompt: str, out_path: str, *, w: int = 1280, h: int = 720, - label: str = "stable diffusion", **_) -> str: - top, bot = _hue_from(prompt) - img = Image.new("RGB", (w, h)) - px = img.load() - for y in range(h): - f = y / max(1, h - 1) - r = int(top[0] + (bot[0] - top[0]) * f) - g = int(top[1] + (bot[1] - top[1]) * f) - b = int(top[2] + (bot[2] - top[2]) * f) - for x in range(w): - px[x, y] = (r, g, b) - d = ImageDraw.Draw(img) - # caption block - title = (prompt or "").split(",")[0].strip()[:120] or "untitled scene" - body = _font(40) - wrapped = textwrap.fill(title, width=34) - d.multiline_text((70, h // 2 - 60), wrapped, font=body, - fill=(233, 220, 196), spacing=10) - tag = _font(22) - d.text((70, h - 70), f"SeeStory placeholder · {label}", font=tag, - fill=(232, 162, 60)) - # thin amber rule - d.rectangle([70, h - 86, 70 + 320, h - 84], fill=(232, 162, 60)) - img.save(out_path, "JPEG", quality=90) - return out_path diff --git a/app/imagegen/stablediffusion.py b/app/imagegen/stablediffusion.py index 1c13ce8..9d8f791 100644 --- a/app/imagegen/stablediffusion.py +++ b/app/imagegen/stablediffusion.py @@ -1,48 +1,55 @@ -""" -Local Stable Diffusion backend (the every-page workhorse). +"""Local Stable Diffusion XL image generation for SeeStory. + +SeeStory deliberately uses repositories that publish a complete Diffusers +``fp16``/Safetensors layout. This matters on the target 8 GB GPU / 16 GB RAM +laptop: Diffusers can stream the component files with low CPU-memory usage and +then offload model components between RAM and CUDA as needed. -Runs on your own GPU via 🤗 diffusers — free, private, no account, and fast -enough to illustrate a whole book. Defaults target an 8 GB laptop card (fp16, -attention slicing, VAE tiling) and a turbo model so each image is a few steps. +* DreamShaper XL Lightning drives every illustrated style, including Cinematic. +* RealVisXL V5 Lightning is used only for the Photorealistic preset. -The pipeline is loaded once, lazily, on first use. If torch/diffusers or the -model aren't available, generate() raises a clear error and the router falls -back to the placeholder backend so the app keeps working. +The former Juggernaut default was removed because its Hub repository currently +mixes a single-file Safetensors checkpoint with ``.bin`` Diffusers components. +Requesting Safetensors through ``from_pretrained`` therefore fails before the +first image is generated. """ +from __future__ import annotations +import gc import os import sys import threading +from typing import Any _LOCK = threading.Lock() _PIPE = None -_PIPE_KEY = None - -# Override via environment (set in run.bat or the UI later). -DEFAULT_MODEL = os.environ.get("SEESTORY_SD_MODEL", "stabilityai/sdxl-turbo") -# Photorealistic model used when the "Photorealistic" art style is chosen. It's a -# Lightning checkpoint, so it stays in the fast few-steps path. Override with -# SEESTORY_SD_PHOTOREAL_MODEL (any diffusers-compatible SDXL repo). -PHOTOREAL_MODEL = os.environ.get("SEESTORY_SD_PHOTOREAL_MODEL", - "SG161222/RealVisXL_V4.0_Lightning") +_PIPE_KEY: str | None = None + +# DreamShaper publishes a proper Diffusers fp16/Safetensors layout and is suited +# to both cinematic and painterly illustration. Keeping one model for all +# illustrated styles also avoids several multi-gigabyte downloads. +DEFAULT_MODEL = os.environ.get( + "SEESTORY_SD_MODEL", "Lykon/dreamshaper-xl-lightning" +) +ARTISTIC_MODEL = os.environ.get( + "SEESTORY_SD_ARTISTIC_MODEL", DEFAULT_MODEL +) +PHOTOREAL_MODEL = os.environ.get( + "SEESTORY_SD_PHOTOREAL_MODEL", "SG161222/RealVisXL_V5.0_Lightning" +) DEFAULT_W = int(os.environ.get("SEESTORY_SD_W", "1024")) DEFAULT_H = int(os.environ.get("SEESTORY_SD_H", "576")) -# Things we never want IN the picture. These go in the model's negative prompt -# (not tacked onto the positive prompt, where CLIP would truncate them on long -# prompts). Override with SEESTORY_SD_NEGATIVE. DEFAULT_NEGATIVE = os.environ.get( "SEESTORY_SD_NEGATIVE", - "text, words, letters, title, book cover, captions, watermark, signature, " - "logo, frame, border, duplicate, cloned face, two faces, extra face, " - "extra head, deformed, disfigured, extra limbs, extra fingers, bad anatomy, " - "blurry, low quality") - -# Negative prompts only bite when classifier-free guidance is on (guidance > 0). -# SDXL-Turbo runs at guidance 0 by design, so negatives are inert there. Set -# SEESTORY_SD_GUIDANCE (e.g. 1.5) to force a little guidance and make the -# negative prompt actually suppress text/watermarks on turbo, at some speed cost. -_GUIDANCE_ENV = os.environ.get("SEESTORY_SD_GUIDANCE") + "text, words, letters, title, book cover, captions, subtitle, watermark, " + "signature, logo, frame, border, low quality, blurry, bad anatomy, bad " + "proportions, deformed body, disfigured, mutation, malformed hands, bad " + "hands, fused fingers, extra fingers, missing fingers, extra arms, extra " + "legs, extra limbs, missing limbs, duplicate limbs, cloned body, duplicate " + "person, extra face, extra head, deformed face, asymmetrical eyes, deformed " + "eyes, deformed mouth", +) def is_available() -> bool: @@ -62,149 +69,290 @@ def has_cuda() -> bool: return False -def _turbo(model: str) -> bool: - m = model.lower() - return "turbo" in m or "lightning" in m or "lcm" in m +def _cuda_total_gb() -> float: + try: + import torch + if torch.cuda.is_available(): + return torch.cuda.get_device_properties(0).total_memory / (1024 ** 3) + except Exception: + pass + return 0.0 + + +def _lightning(model: str) -> bool: + low = model.lower() + return "lightning" in low or "turbo" in low or "lcm" in low + + +def _generation_defaults(model: str) -> tuple[int, float]: + low = model.lower() + if "realvisxl" in low and "lightning" in low: + return 5, 1.8 + if "dreamshaper" in low and "lightning" in low: + return 4, 2.0 + if _lightning(model): + return 6, 2.0 + return 30, 6.0 -def _load(model: str): +def _empty_cache() -> None: + try: + import torch + if torch.cuda.is_available(): + torch.cuda.empty_cache() + try: + torch.cuda.ipc_collect() + except Exception: + pass + except Exception: + pass + + +def _discard_pipe() -> None: + global _PIPE, _PIPE_KEY + _PIPE = None + _PIPE_KEY = None + gc.collect() + _empty_cache() + + +def _from_pretrained( + model: str, kwargs: dict[str, Any], *, allow_download: bool = False +): + """Load a model from cache, optionally allowing the installer to download it. + + Flask/runtime calls always use ``allow_download=False``. Only + ``install_models.py`` opts into network access, keeping multi-gigabyte model + transfers out of the desktop app and making missing-model failures immediate + and actionable. + """ + from diffusers import AutoPipelineForText2Image + + errors: list[Exception] = [] + variants = [dict(kwargs)] + if "variant" in kwargs: + without_variant = dict(kwargs) + without_variant.pop("variant", None) + variants.append(without_variant) + + local_modes = (True, False) if allow_download else (True,) + for local_only in local_modes: + for attempt in variants: + try: + return AutoPipelineForText2Image.from_pretrained( + model, local_files_only=local_only, **attempt + ) + except Exception as exc: + errors.append(exc) + + detail = errors[-1] if errors else "unknown model-loading error" + if not allow_download: + kind = "Photorealistic" if model == PHOTOREAL_MODEL else "Default" + raise RuntimeError( + f"{kind} image model '{model}' is not installed or its cache is " + "incomplete. Close SeeStory and run install_all.bat again. " + "For the Photorealistic style, choose Yes when the installer asks " + f"whether to install that optional model. Technical detail: {detail}" + ) + raise RuntimeError(str(detail)) + + +def _load(model: str, *, allow_download: bool = False): global _PIPE, _PIPE_KEY if _PIPE is not None and _PIPE_KEY == model: return _PIPE + if _PIPE is not None: + _discard_pipe() + import torch - from diffusers import AutoPipelineForText2Image + from diffusers import DPMSolverMultistepScheduler use_cuda = torch.cuda.is_available() dtype = torch.float16 if use_cuda else torch.float32 + kwargs: dict[str, Any] = { + "torch_dtype": dtype, + "use_safetensors": True, + "low_cpu_mem_usage": True, + } + if use_cuda: + kwargs["variant"] = "fp16" + + print(f"[seestory] loading local image model: {model}", file=sys.stderr) + pipe = _from_pretrained(model, kwargs, allow_download=allow_download) + try: - pipe = AutoPipelineForText2Image.from_pretrained( - model, torch_dtype=dtype, variant="fp16" if use_cuda else None, - use_safetensors=True) + low_model = model.lower() + if "realvisxl" in low_model: + pipe.scheduler = DPMSolverMultistepScheduler.from_config( + pipe.scheduler.config, + algorithm_type="sde-dpmsolver++", + use_karras_sigmas=True, + ) + else: + pipe.scheduler = DPMSolverMultistepScheduler.from_config( + pipe.scheduler.config + ) except Exception: - # Many community photoreal checkpoints don't ship a separate "fp16" - # variant — retry without it rather than failing the whole render. - pipe = AutoPipelineForText2Image.from_pretrained( - model, torch_dtype=dtype, use_safetensors=True) + pass + if use_cuda: - pipe = pipe.to("cuda") try: pipe.enable_attention_slicing() - pipe.enable_vae_tiling() except Exception: pass - # Frees VRAM on tight (8 GB) cards by streaming weights from CPU. try: - pipe.enable_model_cpu_offload() + pipe.enable_vae_slicing() + pipe.enable_vae_tiling() except Exception: pass + + # The user's RTX 5060 laptop has 8 GB VRAM. CPU offload is slower than + # placing the whole pipeline on CUDA, but prevents immediate OOM failures. + if _cuda_total_gb() <= 10.5: + try: + pipe.enable_model_cpu_offload() + except Exception: + pipe = pipe.to("cuda") + else: + pipe = pipe.to("cuda") + try: pipe.set_progress_bar_config(disable=True) except Exception: pass + _PIPE, _PIPE_KEY = pipe, model return pipe -def _empty_cache(): - try: - import torch - if torch.cuda.is_available(): - torch.cuda.empty_cache() - torch.cuda.ipc_collect() - except Exception: - pass +def free() -> None: + """Release the current pipeline and as much VRAM/RAM as possible.""" + with _LOCK: + _discard_pipe() -def free(): - """Release the pipeline and free VRAM (used on shutdown / memory pressure).""" - global _PIPE, _PIPE_KEY +def install_and_verify_model(model: str, smoke_path: str | os.PathLike[str]) -> dict[str, float]: + """Download/resume, warm-load, and prove real CUDA inference for a model. + + This is installer-only. The normal Flask path never passes + ``allow_download=True`` and is additionally launched in Hugging Face offline + mode. + """ + if not has_cuda(): + raise RuntimeError("CUDA is required for SeeStory model installation.") + + import torch + with _LOCK: - _PIPE, _PIPE_KEY = None, None - import gc - gc.collect() + pipe = _load(model, allow_download=True) _empty_cache() + try: + torch.cuda.reset_peak_memory_stats() + except Exception: + pass + + steps, guidance = _generation_defaults(model) + generator = torch.Generator(device="cpu").manual_seed(20260802) + prompt = ( + "a single open storybook beside a warm reading lamp, cinematic " + "illustration, one coherent scene, detailed, no people, no text" + ) + with torch.inference_mode(): + result = pipe( + prompt=prompt, + negative_prompt=DEFAULT_NEGATIVE, + num_inference_steps=steps, + guidance_scale=guidance, + height=288, + width=512, + generator=generator, + ) + image = result.images[0] + smoke = os.fspath(smoke_path) + os.makedirs(os.path.dirname(os.path.abspath(smoke)), exist_ok=True) + image.save(smoke, "JPEG", quality=90) + peak_mb = torch.cuda.max_memory_allocated() / (1024 ** 2) + if peak_mb < 32: + raise RuntimeError( + f"Inference returned an image but CUDA use was not proven " + f"(peak allocation only {peak_mb:.1f} MB)." + ) + return {"peak_vram_mb": float(peak_mb)} -def generate(prompt: str, out_path: str, *, model: str = None, - steps: int = None, guidance: float = None, - negative_prompt: str = None, - w: int = None, h: int = None, seed: int = None, **_) -> str: +def generate( + prompt: str, + out_path: str, + *, + model: str | None = None, + steps: int | None = None, + guidance: float | None = None, + negative_prompt: str | None = None, + w: int | None = None, + h: int | None = None, + seed: int | None = None, + **_: Any, +) -> str: if not is_available(): raise RuntimeError( - "Stable Diffusion backend unavailable: install torch + diffusers " - "(see setup.bat / README). Falling back to placeholder.") + "Local image generation is unavailable. Run install_all.bat to " + "install CUDA PyTorch and the image-generation components." + ) + model = model or DEFAULT_MODEL - w = w or DEFAULT_W - h = h or DEFAULT_H + w = int(w or DEFAULT_W) + h = int(h or DEFAULT_H) neg = DEFAULT_NEGATIVE if negative_prompt is None else negative_prompt - with _LOCK: # one generation at a time on the GPU + default_steps, default_guidance = _generation_defaults(model) + steps = int(steps or default_steps) + guidance = float(default_guidance if guidance is None else guidance) + + with _LOCK: try: - pipe = _load(model) - except Exception as e: - # A custom/photoreal checkpoint that can't be fetched or loaded - # shouldn't doom the whole render to placeholders — fall back to the - # known-good default model so we still produce a real image. - if model != DEFAULT_MODEL: - print(f"[seestory] model '{model}' failed to load ({e}); " - f"falling back to {DEFAULT_MODEL}", file=sys.stderr) - model = DEFAULT_MODEL - pipe = _load(model) - else: - raise + pipe = _load(model, allow_download=False) + except Exception as exc: + raise RuntimeError(f"Could not load image model '{model}': {exc}") from exc + import torch - # Resolve guidance: an explicit env override wins; otherwise use the - # value passed in (from the app's "image cleanup" setting) or the default. - if _GUIDANCE_ENV: - try: - guidance = float(_GUIDANCE_ENV) - except ValueError: - pass - if _turbo(model): - # Pure turbo runs at guidance 0, where the negative prompt is ignored - # (text + double-faces slip through). A mild guidance ( >1 ) with a - # couple more steps lets the negative prompt suppress them, still fast. - g = 1.6 if guidance is None else guidance - steps = steps or (4 if g <= 0 else 6) # classic turbo vs turbo+negatives - guidance = g - else: - steps = steps or 28 - guidance = 7.0 if guidance is None else guidance - gen = None + generator = None if seed is not None: - dev = "cuda" if torch.cuda.is_available() else "cpu" - gen = torch.Generator(device=dev).manual_seed(int(seed)) - - # Try at requested size; on out-of-memory, clear the cache and retry at - # progressively smaller sizes so one heavy image never kills the run. - sizes = [(int(w), int(h))] - for fw, fh in ((768, 432), (512, 288)): - if fw < w: + generator = torch.Generator(device="cpu").manual_seed(int(seed)) + + sizes: list[tuple[int, int]] = [(w, h)] + for fw, fh in ((896, 504), (768, 432), (640, 360)): + if fw < w and (fw, fh) not in sizes: sizes.append((fw, fh)) - last_err = None - img = None - for (tw, th) in sizes: + + last_err: Exception | None = None + image = None + for tw, th in sizes: try: - result = pipe(prompt=prompt, negative_prompt=neg, - num_inference_steps=int(steps), - guidance_scale=float(guidance), height=th, - width=tw, generator=gen) - img = result.images[0] + with torch.inference_mode(): + result = pipe( + prompt=prompt, + negative_prompt=neg, + num_inference_steps=steps, + guidance_scale=guidance, + height=th, + width=tw, + generator=generator, + ) + image = result.images[0] break - except torch.cuda.OutOfMemoryError as e: - last_err = e + except torch.cuda.OutOfMemoryError as exc: + last_err = exc + _empty_cache() + except RuntimeError as exc: + if "out of memory" not in str(exc).lower(): + raise + last_err = exc _empty_cache() - continue - except RuntimeError as e: - # CUDA OOM sometimes surfaces as a generic RuntimeError. - if "out of memory" in str(e).lower(): - last_err = e - _empty_cache() - continue - raise - if img is None: - _empty_cache() - raise RuntimeError(f"Stable Diffusion ran out of GPU memory: {last_err}") - - img.save(out_path, "JPEG", quality=92) - _empty_cache() # release VRAM between shots + + if image is None: + raise RuntimeError( + f"GPU memory was exhausted while generating: {last_err}" + ) + + image.save(out_path, "JPEG", quality=94, optimize=True) + _empty_cache() return out_path diff --git a/app/preview_frame.py b/app/preview_frame.py new file mode 100644 index 0000000..4ef7567 --- /dev/null +++ b/app/preview_frame.py @@ -0,0 +1,49 @@ +"""Create a simple local frame used only by the Ken Burns motion preview. + +This is not an image-generation fallback. Real storyboard images always use the +local diffusion model; this frame only gives the motion controls something to +animate before the user has generated a sample image. +""" +from __future__ import annotations + +import hashlib +import textwrap + +from PIL import Image, ImageDraw, ImageFont + + +def _tones(seed: str) -> tuple[tuple[int, int, int], tuple[int, int, int]]: + digest = hashlib.sha256(seed.encode("utf-8")).digest() + top = (28 + digest[0] % 40, 24 + digest[1] % 40, 30 + digest[2] % 50) + bottom = (8 + digest[3] % 24, 6 + digest[4] % 20, 10 + digest[5] % 26) + return top, bottom + + +def _font(size: int): + for name in ("DejaVuSerif.ttf", "DejaVuSans.ttf"): + try: + return ImageFont.truetype(name, size) + except Exception: + continue + return ImageFont.load_default() + + +def create(out_path: str, *, w: int = 1280, h: int = 720) -> str: + top, bottom = _tones("SeeStory motion preview") + img = Image.new("RGB", (w, h)) + draw = ImageDraw.Draw(img) + # Draw the gradient in horizontal bands rather than assigning every pixel in + # Python. This is much faster and is more than smooth enough for a preview. + for y in range(h): + frac = y / max(1, h - 1) + color = tuple(int(a + (b - a) * frac) for a, b in zip(top, bottom)) + draw.line((0, y, w, y), fill=color) + + text = textwrap.fill("Generate a sample image for a real motion preview", width=30) + draw.multiline_text( + (70, h // 2 - 50), text, font=_font(34), fill=(233, 220, 196), spacing=10 + ) + draw.rectangle([70, h - 86, 390, h - 84], fill=(232, 162, 60)) + draw.text((70, h - 70), "SeeStory motion preview", font=_font(22), fill=(232, 162, 60)) + img.save(out_path, "JPEG", quality=90) + return out_path diff --git a/app/server.py b/app/server.py index fb1268e..d5717b3 100644 --- a/app/server.py +++ b/app/server.py @@ -3,20 +3,18 @@ Pipeline: ebook + Parroty MP3 + Parroty timestamps -> chapters (same parser as Parroty) mapped onto the audio timeline - -> shots (one image/clip each), prompted + routed by the director - -> images (Stable Diffusion / Copilot / placeholder) + -> shots (one image/clip each), prompted by the director + -> local GPU-generated images -> Ken Burns motion clips -> one chaptered MP4 synced to the narration. Runs locally at http://127.0.0.1:5001 so it sits beside Parroty (port 5000). """ -import io import json import os import random import sys - import shutil import threading import time @@ -35,6 +33,8 @@ from . import assembler as ASM from . import pagemap from . import subtitles as SUB +from . import desktop_runtime as DESKTOP +from . import validation as VALID COVER_SECONDS = 6.0 # how long the book cover holds at the very start @@ -47,12 +47,27 @@ app = Flask(__name__) app.config["MAX_CONTENT_LENGTH"] = 5 * 1024 * 1024 * 1024 # 5 GB (long audiobooks) +# This is a local desktop app that is upgraded in-place. A persistent Chrome/Edge +# app profile is useful for clean startup, but browser caching must never leave an +# old HTML/JS interface visible after an upgrade. +app.config["SEND_FILE_MAX_AGE_DEFAULT"] = 0 + + +@app.after_request +def _disable_ui_cache(response): + if request.path == "/" or request.path.startswith("/static/"): + response.headers["Cache-Control"] = "no-store, no-cache, must-revalidate, max-age=0" + response.headers["Pragma"] = "no-cache" + response.headers["Expires"] = "0" + return response PORT = int(os.environ.get("SEESTORY_PORT", "5001")) VIDEO_W = int(os.environ.get("SEESTORY_W", "1280")) VIDEO_H = int(os.environ.get("SEESTORY_H", "720")) VIDEO_FPS = int(os.environ.get("SEESTORY_FPS", "30")) +DESKTOP.install(app, base_dir=BASE, port=PORT) + # ── helpers ────────────────────────────────────────────────────────────── class _PairedChapter: @@ -71,9 +86,22 @@ def _slug(s, n=40): def _job_dir(job): + """Internal job folder helper for server-created safe job names.""" return os.path.join(OUTPUT, job) +def _session_dir_safe(job): + return VALID.safe_session_dir(OUTPUT, job) + + +def _temp_upload_path(prefix: str, filename: str) -> str: + return VALID.temp_upload_path(UPLOADS, prefix, filename) + + +def _clean_motion(raw) -> dict: + return VALID.clean_motion(raw, KB.DEFAULT_MOTION) + + def _sse(obj): return f"data: {json.dumps(obj)}\n\n" @@ -86,11 +114,17 @@ def save_project(proj): def load_project(job): - p = os.path.join(_job_dir(job), "project.json") + folder = _session_dir_safe(job) + if not folder: + return None + p = os.path.join(folder, "project.json") if not os.path.exists(p): return None - with open(p, encoding="utf-8") as f: - return json.load(f) + try: + with open(p, encoding="utf-8") as f: + return json.load(f) + except (OSError, json.JSONDecodeError): + return None def _parse_book(path): @@ -113,9 +147,8 @@ def _shot_from(d): s = TL.Shot(**{k: d[k] for k in ( "id", "chapter_index", "chapter_title", "shot_in_chapter", "page_start", "page_end", "text", "start_ms", "end_ms")}) - for k in ("is_chapter_start", "word_count", "prompt", "backend", - "highlight_score", "highlighted", "image_path", "status", - "error", "motion"): + for k in ("is_chapter_start", "word_count", "prompt", "image_path", + "status", "error", "motion"): if k in d: setattr(s, k, d[k]) return s @@ -129,13 +162,19 @@ def _shot_json(s): @app.route("/") def index(): return render_template( - "index.html", ffmpeg_ok=ASM.ensure_ffmpeg(), probe=imagegen.probe(), - presets=list(KB.PRESETS.keys()), styles=list(DIR.StyleBible.PRESETS.keys()), - default_motion=KB.DEFAULT_MOTION) + "index.html", presets=list(KB.PRESETS.keys()), + styles=list(DIR.StyleBible.PRESETS.keys()), default_motion=KB.DEFAULT_MOTION) + + +@app.route("/api/health") +def api_health(): + """Lightweight launcher readiness check; avoids loading GPU libraries.""" + return jsonify({"ok": True}) @app.route("/api/probe") def api_probe(): + """Detailed local diagnostics, intentionally separate from startup readiness.""" return jsonify(imagegen.probe() | {"ffmpeg": ASM.ensure_ffmpeg()}) @@ -146,7 +185,7 @@ def page_check(): ebook = request.files.get("ebook") if not ebook: return jsonify({"has_pages": False}) - tmp = os.path.join(UPLOADS, "_pagecheck" + os.path.splitext(ebook.filename)[1].lower()) + tmp = _temp_upload_path("pagecheck", ebook.filename) ebook.save(tmp) info = {"has_pages": False, "page_count": 0} try: @@ -170,76 +209,76 @@ def page_check(): @app.route("/api/sample", methods=["POST"]) def sample(): - """Generate ONE preview image from a random page, using Stable Diffusion - (falls back to placeholder if SD isn't available).""" + """Generate one local preview image from a random page.""" ebook = request.files.get("ebook") if not ebook: return jsonify({"error": "Add your ebook above first, then generate a sample."}), 400 + style_key = request.form.get("style_key", "cinematic") custom_style = request.form.get("custom_style", "") - wpp = int(request.form.get("words_per_page", 280) or 280) - - tmp = os.path.join(UPLOADS, "_sample" + os.path.splitext(ebook.filename)[1].lower()) - ebook.save(tmp) try: - book = _parse_book(tmp) - except Exception as e: - return jsonify({"error": f"Could not read the ebook: {e}"}), 400 - finally: - pass - - chapters = [c for c in book.chapters if len((c.text or "").split()) >= 30] or book.chapters - if not chapters: - return jsonify({"error": "No readable text found in the ebook."}), 400 - ch = random.choice(chapters) - words = (ch.text or "").split() - if len(words) > wpp: - start = random.randint(0, len(words) - wpp) - page_text = " ".join(words[start:start + wpp]) - page_no = start // max(1, wpp) + 1 - else: - page_text = " ".join(words) - page_no = 1 - - shot = TL.Shot(id="sample", chapter_index=0, chapter_title=ch.title, - shot_in_chapter=0, page_start=0, page_end=0, text=page_text, - start_ms=0, end_ms=1000) - bible = DIR.StyleBible(style_key, custom_style) - DIR.direct([shot], bible) - shot.backend = "stablediffusion" if imagegen.probe()["stablediffusion"] else "placeholder" - - sdir = os.path.join(OUTPUT, "_sample") - os.makedirs(sdir, exist_ok=True) - fn = f"sample_{int(time.time())}.jpg" - # Preview with the SAME model + guidance the real build will use, so the - # sample actually reflects the chosen style (e.g. the photoreal model). - sd_opts = {"w": 1024, "h": 576} - try: - sd_opts["guidance"] = float(request.form.get("sd_guidance", 1.6)) + wpp = max(80, min(2000, int(request.form.get("words_per_page", 280) or 280))) except (TypeError, ValueError): - pass - if style_key == "photoreal" and not os.environ.get("SEESTORY_SD_MODEL"): - sd_opts["model"] = imagegen.stablediffusion.PHOTOREAL_MODEL + wpp = 280 + + tmp = _temp_upload_path("sample", ebook.filename) + ebook.save(tmp) try: - res = imagegen.generate_for(shot, os.path.join(sdir, fn), sd_opts=sd_opts) - except Exception as e: - return jsonify({"error": f"Sample generation failed: {e}"}), 500 + try: + book = _parse_book(tmp) + except Exception as exc: + return jsonify({"error": f"Could not read the ebook: {exc}"}), 400 + + chapters = [c for c in book.chapters if len((c.text or "").split()) >= 30] or book.chapters + if not chapters: + return jsonify({"error": "No readable text found in the ebook."}), 400 + + ch = random.choice(chapters) + words = (ch.text or "").split() + if len(words) > wpp: + word_start = random.randint(0, len(words) - wpp) + page_text = " ".join(words[word_start:word_start + wpp]) + page_no = word_start // max(1, wpp) + 1 + else: + page_text = " ".join(words) + page_no = 1 + + shot = TL.Shot( + id="sample", chapter_index=0, chapter_title=ch.title, shot_in_chapter=0, + page_start=0, page_end=0, text=page_text, start_ms=0, end_ms=1000, + ) + DIR.direct([shot], DIR.StyleBible(style_key, custom_style)) + + sdir = os.path.join(OUTPUT, "_sample") + os.makedirs(sdir, exist_ok=True) + fn = f"sample_{int(time.time())}.jpg" + opts = {"w": 1024, "h": 576} + if not os.environ.get("SEESTORY_SD_MODEL"): + SD = imagegen.stablediffusion + if style_key == "photoreal": + opts["model"] = SD.PHOTOREAL_MODEL + elif style_key == "cinematic": + opts["model"] = SD.DEFAULT_MODEL + else: + opts["model"] = SD.ARTISTIC_MODEL + try: + imagegen.generate_for(shot, os.path.join(sdir, fn), sd_opts=opts) + except Exception as exc: + return jsonify({"error": f"Sample generation failed: {exc}"}), 500 + + return jsonify({ + "image_url": f"/image/_sample/{fn}", + "page_text": page_text, + "prompt": shot.prompt, + "chapter_title": ch.title, + "page_no": page_no, + }) finally: try: os.unlink(tmp) except OSError: pass - return jsonify({ - "image_url": f"/image/_sample/{fn}", - "page_text": page_text, - "prompt": shot.prompt, - "chapter_title": ch.title, - "page_no": page_no, - "backend_used": res["backend"], - "note": res.get("note", ""), - }) - @app.route("/api/ingest", methods=["POST"]) def ingest(): @@ -350,19 +389,23 @@ def stream(): use_chapters.append(_PairedChapter(mtitle, txt)) spans = _spans(markers, total_ms) - words_per_page = int(opts.get("words_per_page", 280)) - pages_per_shot = int(opts.get("pages_per_shot", 1)) - mode = opts.get("mode", "both") + try: + words_per_page = max(80, min(2000, int(opts.get("words_per_page", 280)))) + except (TypeError, ValueError): + words_per_page = 280 + try: + pages_per_shot = max(1, min(20, int(opts.get("pages_per_shot", 1)))) + except (TypeError, ValueError): + pages_per_shot = 1 style_key = opts.get("style_key", "cinematic") - custom_style = opts.get("custom_style", "") - copilot_every = int(opts.get("copilot_every_pages", 10)) - copilot_cap = int(opts.get("copilot_cap", 30)) - page_basis = opts.get("page_basis", "words") - page_count = int(opts.get("page_count", 0) or 0) + if style_key not in DIR.StyleBible.PRESETS: + style_key = "cinematic" + custom_style = (opts.get("custom_style", "") or "")[:500] + page_basis = "embedded" if opts.get("page_basis") == "embedded" else "words" try: - guidance = float(opts.get("sd_guidance", 1.6)) + page_count = max(0, min(1_000_000, int(opts.get("page_count", 0) or 0))) except (TypeError, ValueError): - guidance = 1.6 + page_count = 0 yield _sse({"type": "stage", "pct": 58, "label": "Splitting into pages…"}) shots = TL.segment_book(use_chapters, spans, words_per_page=words_per_page, @@ -372,20 +415,13 @@ def stream(): bible = DIR.StyleBible(style_key, custom_style) DIR.direct(shots, bible) - yield _sse({"type": "stage", "pct": 88, "label": "Choosing image sources…"}) - sd_backend = "stablediffusion" if imagegen.probe()["stablediffusion"] else "placeholder" - summary = DIR.route_backends( - shots, mode=mode, sd_backend=sd_backend, - copilot_every_pages=copilot_every, copilot_cap=copilot_cap) - default_motion = dict(KB.DEFAULT_MOTION) + yield _sse({"type": "stage", "pct": 88, "label": "Preparing the storyboard…"}) try: - m = json.loads(opts.get("motion", "") or "{}") - if isinstance(m, dict): - default_motion.update({k: m[k] for k in m if k in KB.DEFAULT_MOTION}) - except Exception: - pass - for s in shots: - s.motion = dict(default_motion) + default_motion = _clean_motion(json.loads(opts.get("motion", "") or "{}")) + except (TypeError, ValueError, json.JSONDecodeError): + default_motion = dict(KB.DEFAULT_MOTION) + for shot in shots: + shot.motion = dict(default_motion) proj = { "job": job, "title": book.title or "Audiobook", @@ -396,11 +432,9 @@ def stream(): "total_ms": total_ms, "markers": markers, "settings": { - "mode": mode, "words_per_page": words_per_page, + "words_per_page": words_per_page, "pages_per_shot": pages_per_shot, "style_key": style_key, - "custom_style": custom_style, "copilot_every_pages": copilot_every, - "copilot_cap": copilot_cap, "sd_backend": sd_backend, - "guidance": guidance, + "custom_style": custom_style, "page_basis": page_basis, "page_count": page_count, "subtitle_mode": opts.get("subtitle_mode", "none"), "subtitle_file": os.path.basename(subtitle_path) if subtitle_path else None, @@ -409,7 +443,6 @@ def stream(): }, "bible": bible.to_json(), "shots": [_shot_json(s) for s in shots], - "routing": summary, "alignment": { "skipped": align["skipped"], "empty_text": empty_text, @@ -440,13 +473,13 @@ def update_shot(job, shot_id): body = request.get_json(force=True) for s in proj["shots"]: if s["id"] == shot_id: - for k in ("prompt", "backend", "motion"): - if k in body: - s[k] = body[k] if "prompt" in body: + s["prompt"] = str(body.get("prompt") or "")[:2000] # Remember the user hand-edited this prompt, so a later regenerate # won't overwrite it with an auto-rebuilt one. s["prompt_edited"] = True + if "motion" in body: + s["motion"] = _clean_motion(body.get("motion")) save_project(proj) return jsonify(s) return jsonify({"error": "shot not found"}), 404 @@ -469,10 +502,11 @@ def delete_shot(job, shot_id): shots[idx - 1]["end_ms"] = gone["end_ms"] elif shots: shots[idx]["start_ms"] = gone["start_ms"] + folder = _session_dir_safe(job) img = gone.get("image_path") - if img and os.path.exists(os.path.join(_job_dir(job), img)): + if folder and img and os.path.exists(os.path.join(folder, img)): try: - os.unlink(os.path.join(_job_dir(job), img)) + os.unlink(os.path.join(folder, img)) except OSError: pass save_project(proj) @@ -503,36 +537,39 @@ def regenerate_shot(job, shot_id): rec["prompt"] = s.prompt except Exception: pass - out = os.path.join(_job_dir(job), "images", f"{shot_id}.jpg") - # A manual regenerate should always give the intended backend a fresh try — - # clear any per-run Copilot breaker left over from a bulk run. - if rec.get("backend") == "copilot": - try: - from .imagegen import copilot_backend as _cb - _cb.reset_run_state() - except Exception: - pass - res = imagegen.generate_for(s, out, sd_opts=_sd_opts(proj)) + folder = _session_dir_safe(job) + if not folder: + return jsonify({"error": "Project folder is missing."}), 404 + out = os.path.join(folder, "images", f"{shot_id}.jpg") + try: + imagegen.generate_for(s, out, sd_opts=_sd_opts(proj)) + except Exception as exc: + rec["status"] = "error" + rec["error"] = str(exc) + save_project(proj) + return jsonify({"error": str(exc)}), 500 rec["image_path"] = f"images/{shot_id}.jpg" rec["status"] = "done" - rec["backend_used"] = res["backend"] - rec["error"] = res.get("error", "") + rec["error"] = "" save_project(proj) - return jsonify(rec | {"cache_bust": int(time.time()), "note": res.get("note", "")}) + return jsonify(rec | {"cache_bust": int(time.time())}) def _sd_opts(proj): - s = proj["settings"] - o = {"w": s.get("w", VIDEO_W), "h": s.get("h", VIDEO_H)} - g = s.get("guidance") - if g is not None: - o["guidance"] = g - # Photorealistic style → load the photoreal checkpoint instead of turbo, - # unless the user has pinned a specific model via SEESTORY_SD_MODEL. - if s.get("style_key") == "photoreal" and not os.environ.get("SEESTORY_SD_MODEL"): - from .imagegen import stablediffusion as SD - o["model"] = SD.PHOTOREAL_MODEL - return o + settings = proj["settings"] + opts = {"w": settings.get("w", VIDEO_W), "h": settings.get("h", VIDEO_H)} + # An explicit SEESTORY_SD_MODEL pins every style to one user-selected model. + if os.environ.get("SEESTORY_SD_MODEL"): + return opts + from .imagegen import stablediffusion as SD + style = settings.get("style_key", "cinematic") + if style == "photoreal": + opts["model"] = SD.PHOTOREAL_MODEL + elif style == "cinematic": + opts["model"] = SD.DEFAULT_MODEL + else: + opts["model"] = SD.ARTISTIC_MODEL + return opts @app.route("/api/project//generate", methods=["POST"]) @@ -543,62 +580,42 @@ def generate_all(job): @stream_with_context def stream(): + folder = _session_dir_safe(job) + if not folder: + yield _sse({"type": "error", "message": "Project folder is missing."}) + return sd_opts = _sd_opts(proj) shots = proj["shots"] - pending = [s for s in shots if s.get("status") != "done" - or not s.get("image_path")] + pending = [shot for shot in shots + if shot.get("status") != "done" or not shot.get("image_path")] yield _sse({"type": "start", "total": len(pending), "already": len(shots) - len(pending)}) - # If any shot is meant to use Copilot, reset its per-run state and report - # its status once up front so any fallback reason is visible immediately. - if any(s.get("backend") == "copilot" for s in pending): - try: - from .imagegen import copilot_backend as _cb - _cb.reset_run_state() - msg = _cb.diagnose() - sys.stderr.write(f"[seestory] copilot: {msg}\n") - sys.stderr.flush() - yield _sse({"type": "info", "message": f"Copilot: {msg}"}) - except Exception: - pass - done = 0 - warned_copilot = False + failed = 0 for rec in shots: if rec.get("status") == "done" and rec.get("image_path"): continue - s = _shot_from(rec) - out = os.path.join(_job_dir(job), "images", f"{rec['id']}.jpg") + shot = _shot_from(rec) + out = os.path.join(folder, "images", f"{rec['id']}.jpg") try: - res = imagegen.generate_for(s, out, sd_opts=sd_opts) + imagegen.generate_for(shot, out, sd_opts=sd_opts) rec["image_path"] = f"images/{rec['id']}.jpg" rec["status"] = "done" - rec["backend_used"] = res["backend"] - rec["error"] = res.get("error", "") - note = res.get("note", "") - except Exception as e: + rec["error"] = "" + except Exception as exc: + failed += 1 rec["status"] = "error" - rec["error"] = str(e) - note = "error" - # The first time a Copilot shot falls back, surface WHY prominently - # (once) in the build log + console, so it isn't lost in the per-shot - # noise. - if (not warned_copilot and rec.get("backend") == "copilot" - and rec.get("backend_used") not in (None, "", "copilot")): - warned_copilot = True - detail = rec.get("error") or "Copilot was unavailable." - sys.stderr.write(f"[seestory] copilot fallback: {detail}\n") - sys.stderr.flush() - yield _sse({"type": "info", "message": detail}) + rec["error"] = str(exc) + rec["image_path"] = None done += 1 save_project(proj) - yield _sse({"type": "shot", "id": rec["id"], "done": done, - "total": len(pending), "status": rec["status"], - "backend_used": rec.get("backend_used", ""), - "note": note, "error": rec.get("error", ""), - "cache_bust": int(time.time())}) - yield _sse({"type": "complete", "done": done}) + yield _sse({ + "type": "shot", "id": rec["id"], "done": done, + "total": len(pending), "status": rec["status"], + "error": rec.get("error", ""), "cache_bust": int(time.time()), + }) + yield _sse({"type": "complete", "done": done, "failed": failed}) return Response(stream(), mimetype="text/event-stream") @@ -611,11 +628,21 @@ def assemble(job): @stream_with_context def stream(): - jd = _job_dir(job) + jd = _session_dir_safe(job) + if not jd: + yield _sse({"type": "error", "message": "Project folder is missing."}) + return s = proj["settings"] w, h, fps = s.get("w", VIDEO_W), s.get("h", VIDEO_H), s.get("fps", VIDEO_FPS) - ready = [r for r in proj["shots"] if r.get("image_path") - and os.path.exists(os.path.join(jd, r["image_path"]))] + ready = [rec for rec in proj["shots"] if rec.get("image_path") + and os.path.exists(os.path.join(jd, rec["image_path"]))] + missing = len(proj["shots"]) - len(ready) + if missing: + yield _sse({ + "type": "error", + "message": f"{missing} storyboard image(s) are missing. Generate or regenerate them before stitching." + }) + return if not ready: yield _sse({"type": "error", "message": "No images yet — generate first."}) return @@ -718,18 +745,6 @@ def prog(frac): return Response(stream(), mimetype="text/event-stream") -def _session_dir_safe(job): - """Resolve a job to its folder under OUTPUT, or None — guards against path - traversal and non-session folders (_sample, _ingest, .gitkeep).""" - name = os.path.basename(os.path.normpath(job or "")) - if not name or name.startswith("_") or name.startswith("."): - return None - d = os.path.join(OUTPUT, name) - if os.path.dirname(os.path.abspath(d)) != os.path.abspath(OUTPUT): - return None - return d if os.path.isdir(d) else None - - @app.route("/api/project/", methods=["DELETE"]) def delete_project(job): """Remove a single recent session (its folder, images and any built video).""" @@ -743,14 +758,6 @@ def delete_project(job): return jsonify({"ok": True}) -@app.route("/api/copilot_test", methods=["POST"]) -def copilot_test(): - """Fire one real Copilot call and report whether it worked + why not.""" - from .imagegen import copilot_backend as cb - ok, detail = cb.test() - return jsonify({"ok": ok, "detail": detail}) - - @app.route("/api/projects/clear", methods=["POST"]) def clear_projects(): """Remove all recent sessions at once.""" @@ -778,13 +785,9 @@ def motion_all(job): proj = load_project(job) if not proj: return jsonify({"error": "Project not found."}), 404 - m = (request.get_json(force=True) or {}).get("motion") or {} - clean = {k: m[k] for k in m if k in KB.DEFAULT_MOTION} - for s in proj["shots"]: - mm = dict(KB.DEFAULT_MOTION) - mm.update(s.get("motion") or {}) - mm.update(clean) - s["motion"] = mm + motion = _clean_motion((request.get_json(force=True) or {}).get("motion")) + for shot in proj["shots"]: + shot["motion"] = dict(motion) save_project(proj) return jsonify({"ok": True, "count": len(proj["shots"])}) @@ -818,7 +821,8 @@ def list_projects(): "job": p.get("job", name), "title": p.get("title", "Audiobook"), "shots": len(shots), "done": done, - "has_video": bool(p.get("video_file")), + "has_video": bool(p.get("video_file") and + os.path.exists(os.path.join(OUTPUT, name, p["video_file"]))), "video_file": p.get("video_file"), "total_ms": p.get("total_ms", 0), "modified": modified, @@ -829,10 +833,11 @@ def list_projects(): @app.route("/api/motion_preview", methods=["POST"]) def motion_preview(): - """Render a short Ken Burns clip so the user can see the motion before - committing. Uses the latest sample image if there is one, else a placeholder.""" + """Render a short Ken Burns clip so the user can see motion before committing. + The latest generated sample is used when available; otherwise a neutral local + preview frame is created solely for this animation preview.""" body = request.get_json(force=True) or {} - motion = body.get("motion") or {} + motion = _clean_motion(body.get("motion")) sdir = os.path.join(OUTPUT, "_sample") os.makedirs(sdir, exist_ok=True) imgs = sorted(f for f in os.listdir(sdir) @@ -840,10 +845,9 @@ def motion_preview(): if imgs: src = os.path.join(sdir, imgs[-1]) else: - from .imagegen import placeholder + from . import preview_frame src = os.path.join(sdir, "preview_src.jpg") - placeholder.generate("a sweeping landscape, motion preview", src, - w=1024, h=576, label="preview") + preview_frame.create(src, w=1024, h=576) out = os.path.join(sdir, f"preview_{int(time.time())}.mp4") # Length follows the drift pace (set by speed) plus a short hold, so the # preview shows the motion completing and settling — exactly the real look. @@ -859,26 +863,24 @@ def motion_preview(): @app.route("/image//") def image(job, fn): - return send_from_directory(os.path.join(_job_dir(job)), fn) + if job == "_sample": + folder = os.path.join(OUTPUT, "_sample") + else: + folder = _session_dir_safe(job) + if not folder: + return jsonify({"error": "not found"}), 404 + return send_from_directory(folder, fn) @app.route("/download//") def download(job, fn): - return send_from_directory(_job_dir(job), fn, as_attachment=True) + folder = _session_dir_safe(job) + if not folder: + return jsonify({"error": "not found"}), 404 + return send_from_directory(folder, fn, as_attachment=True) # ── startup ────────────────────────────────────────────────────────────── -def _raise_priority(): - """Best-effort: keep GPU work from being throttled when the window is in - the background (mirrors the issue Parroty hit on laptops).""" - try: - if os.name == "nt": - import ctypes - ctypes.windll.kernel32.SetPriorityClass(-1, 0x00000080) # HIGH - except Exception: - pass - - def _open_browser(): time.sleep(1.2) try: @@ -888,7 +890,7 @@ def _open_browser(): def main(): - _raise_priority() + DESKTOP.start() if "--no-browser" not in sys.argv: threading.Thread(target=_open_browser, daemon=True).start() print(f"SeeStory running at http://127.0.0.1:{PORT}") diff --git a/app/static/app.js b/app/static/app.js index e952e75..353c87d 100644 --- a/app/static/app.js +++ b/app/static/app.js @@ -8,7 +8,6 @@ const $$ = (s, r = document) => [...r.querySelectorAll(s)]; const CFG = window.SEESTORY; let PROJECT = null; // the live project object from the server -let MODE = "both"; /* ── tiny helpers ──────────────────────────────────────────────────────── */ function fmtMs(ms) { @@ -29,8 +28,15 @@ function fmtWhen(sec) { } function show(id) { $("#" + id).classList.remove("hidden"); } function hide(id) { $("#" + id).classList.add("hidden"); } +function escapeHtml(value) { + return String(value ?? "").replace(/[&<>"']/g, ch => ({ + "&": "&", "<": "<", ">": ">", '"': """, "'": "'" + })[ch]); +} +function jobPart() { return encodeURIComponent(PROJECT?.job || ""); } function imgUrl(path, bust) { - return `/image/${PROJECT.job}/${path}` + (bust ? `?t=${bust}` : ""); + const safePath = String(path || "").split("/").map(encodeURIComponent).join("/"); + return `/image/${jobPart()}/${safePath}` + (bust ? `?t=${encodeURIComponent(bust)}` : ""); } /* build/error log shared by the generate + assemble steps */ @@ -71,7 +77,12 @@ async function readSSE(resp, onEvent) { while ((i = buf.indexOf("\n\n")) >= 0) { const chunk = buf.slice(0, i); buf = buf.slice(i + 2); const line = chunk.split("\n").find(l => l.startsWith("data:")); - if (line) { try { onEvent(JSON.parse(line.slice(5).trim())); } catch {} } + if (line) { + let event; + try { event = JSON.parse(line.slice(5).trim()); } + catch { continue; } + onEvent(event); + } } } } @@ -139,13 +150,13 @@ async function checkPages() { } } -/* sample image preview (Stable Diffusion) */ +/* sample image preview */ async function generateSample() { if (!files.ebook) { $("#sample-hint").textContent = "Add the ebook above first."; return; } show("sample-panel"); const wrap = $("#sample-imgwrap"); wrap.innerHTML = `

`; - $("#sample-meta").textContent = "Generating… (first run loads the model)"; + $("#sample-meta").textContent = "Generating sample…"; $("#sample-page").textContent = ""; $("#sample-prompt").textContent = ""; try { const fd = new FormData(); @@ -156,15 +167,12 @@ async function generateSample() { const r = await fetch("/api/sample", { method: "POST", body: fd }); const d = await r.json(); if (!r.ok) throw new Error(d.error || "Sample failed."); - const src = d.backend_used === "stablediffusion" ? "Stable Diffusion" - : d.backend_used === "copilot" ? "Copilot" : "placeholder"; - wrap.innerHTML = ``; - $("#sample-meta").innerHTML = - `${d.chapter_title} · ~page ${d.page_no} · ${src}`; + wrap.innerHTML = ``; + $("#sample-meta").innerHTML = `${escapeHtml(d.chapter_title)} · ~page ${Number(d.page_no) || 1}`; $("#sample-page").textContent = d.page_text; $("#sample-prompt").textContent = "Prompt: " + d.prompt; } catch (e) { - wrap.innerHTML = `${e.message}`; + wrap.innerHTML = `${escapeHtml(e.message)}`; $("#sample-meta").textContent = ""; } } @@ -177,14 +185,6 @@ function refreshIngestBtn() { } refreshIngestBtn(); -$$("#mode-cards .mode-card").forEach(card => card.addEventListener("click", () => { - $$("#mode-cards .mode-card").forEach(c => c.classList.remove("sel")); - card.classList.add("sel"); - MODE = card.dataset.mode; - const copOff = MODE === "sd_only"; - $("#cop-every-wrap").style.opacity = copOff ? .4 : 1; - $("#cop-cap-wrap").style.opacity = copOff ? .4 : 1; -})); $("#btn-ingest").addEventListener("click", async () => { const btn = $("#btn-ingest"); @@ -197,15 +197,14 @@ $("#btn-ingest").addEventListener("click", async () => { fd.append("audio", files.audio); if (files.ts) fd.append("timestamps", files.ts); fd.append("timestamps_text", $("#ts-paste").value); - fd.append("mode", MODE); fd.append("page_basis", PAGE_BASIS); fd.append("page_count", PAGE_COUNT); fd.append("motion", JSON.stringify(getGlobalMotion())); if (files.cover) fd.append("cover", files.cover); if (files.subtitle) fd.append("subtitle", files.subtitle); fd.append("subtitle_mode", $("#subtitle_mode").value); - ["words_per_page", "pages_per_shot", "style_key", "custom_style", - "copilot_every_pages", "copilot_cap", "sd_guidance"].forEach(k => fd.append(k, $("#" + k).value)); + ["words_per_page", "pages_per_shot", "style_key", "custom_style"] + .forEach(k => fd.append(k, $("#" + k).value)); let project = null, ingestErr = null; try { @@ -299,8 +298,6 @@ $("#btn-motion-preview").addEventListener("click", async () => { }); /* ── STEP 2 · storyboard ───────────────────────────────────────────────── */ -const BACKENDS = [["stablediffusion", "SD"], ["copilot", "Copilot"], ["placeholder", "Plain"]]; - function motionControls(shot) { const m = Object.assign({}, CFG.defaultMotion, shot.motion || {}); const TIPS = { @@ -338,8 +335,7 @@ function motionControls(shot) { function shotCard(shot) { const el = document.createElement("div"); - el.className = "shot" + (shot.is_chapter_start ? " chapter-start" : "") + - (shot.backend === "copilot" ? " is-copilot" : ""); + el.className = "shot" + (shot.is_chapter_start ? " chapter-start" : ""); el.dataset.id = shot.id; const thumb = shot.image_path ? `` @@ -348,16 +344,10 @@ function shotCard(shot) {
${thumb} ${fmtMs(shot.start_ms)}–${fmtMs(shot.end_ms)} - ${shot.backend === "stablediffusion" ? "SD" : shot.backend}
-
${shot.chapter_title} · shot ${shot.shot_in_chapter + 1}${shot.highlighted ? " · ★ highlight" : ""}
- -
-
- ${BACKENDS.map(([v, t]) => ``).join("")} -
-
+
${escapeHtml(shot.chapter_title)} · shot ${shot.shot_in_chapter + 1}
+ ${motionControls(shot)}
@@ -370,26 +360,12 @@ function shotCard(shot) { function wireShot(el, shot) { const id = shot.id; - // prompt save on blur const ta = $(".shot-prompt", el); ta.addEventListener("blur", () => saveShot(id, { prompt: ta.value })); - // backend segmented control - $$(".seg[data-seg=backend] button", el).forEach(b => b.addEventListener("click", () => { - $$(".seg[data-seg=backend] button", el).forEach(x => x.classList.remove("on")); - b.classList.add("on"); - const v = b.dataset.v; - findShot(id).backend = v; - const badge = $(".tbk", el); - badge.className = "tbk " + v; badge.textContent = v === "stablediffusion" ? "SD" : v; - el.classList.toggle("is-copilot", v === "copilot"); - saveShot(id, { backend: v }); - })); - // motion toggle $(".m-toggle", el).addEventListener("click", () => { const m = $(".motion", el); m.classList.toggle("open"); $(".m-toggle", el).textContent = (m.classList.contains("open") ? "▾" : "▸") + " motion"; }); - // motion inputs $$(".motion [data-m]", el).forEach(inp => { const out = inp.parentElement.querySelector("output"); inp.addEventListener("input", () => { if (out) out.textContent = inp.value; }); @@ -435,7 +411,7 @@ function findShot(id) { return PROJECT.shots.find(s => s.id === id); } async function saveShot(id, patch) { try { - await fetch(`/api/project/${PROJECT.job}/shot/${id}`, { + await fetch(`/api/project/${jobPart()}/shot/${encodeURIComponent(id)}`, { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify(patch) }); @@ -447,25 +423,23 @@ async function regen(el, id) { thumb.innerHTML = `
` + `${fmtMs(findShot(id).start_ms)}–${fmtMs(findShot(id).end_ms)}`; try { - const r = await fetch(`/api/project/${PROJECT.job}/shot/${id}/regenerate`, { method: "POST" }); + const r = await fetch(`/api/project/${jobPart()}/shot/${encodeURIComponent(id)}/regenerate`, { method: "POST" }); const rec = await r.json(); + if (!r.ok) throw new Error(rec.error || "Regenerate failed."); Object.assign(findShot(id), rec); thumb.innerHTML = `` + - `${fmtMs(rec.start_ms)}–${fmtMs(rec.end_ms)}` + - `${(rec.backend_used || rec.backend) === "stablediffusion" ? "SD" : (rec.backend_used || rec.backend)}`; - if (rec.backend === "copilot" && rec.backend_used && rec.backend_used !== "copilot") { - toast(rec.note || rec.error || `Copilot unavailable — used ${rec.backend_used} instead.`, "warn"); - } + `${fmtMs(rec.start_ms)}–${fmtMs(rec.end_ms)}`; } catch (e) { - thumb.innerHTML = `regenerate failed`; + thumb.innerHTML = `${escapeHtml(e.message || "regenerate failed")}`; + toast(e.message || "Regenerate failed.", "warn"); } } async function del(el, id) { if (!confirm("Delete this shot? Its on-screen time is handed to the shot before it.")) return; try { - const r = await fetch(`/api/project/${PROJECT.job}/shot/${id}/delete`, { method: "POST" }); + const r = await fetch(`/api/project/${jobPart()}/shot/${encodeURIComponent(id)}/delete`, { method: "POST" }); const data = await r.json(); if (data.shots) PROJECT.shots = data.shots; el.remove(); @@ -490,9 +464,8 @@ function renderBoard() { } function updateBoardSummary() { const n = PROJECT.shots.length; - const cop = PROJECT.shots.filter(s => s.backend === "copilot").length; const dur = PROJECT.total_ms; - let s = `${n} shots · ${cop} premium (Copilot) · ${fmtMs(dur)} runtime`; + let s = `${n} shots · ${fmtMs(dur)} runtime`; const a = PROJECT.alignment; if (a) { s += ` · aligned to your chapter list`; @@ -516,7 +489,7 @@ $("#btn-apply-motion-all").addEventListener("click", () => { } }); }); - fetch(`/api/project/${PROJECT.job}/motion_all`, { + fetch(`/api/project/${jobPart()}/motion_all`, { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ motion: m }) }).catch(() => {}); @@ -548,39 +521,50 @@ async function runGenerate() { logLine(`Generating ${PROJECT.shots.length} shots…`); let completed = false; try { - const r = await fetch(`/api/project/${PROJECT.job}/generate`, { method: "POST" }); + const r = await fetch(`/api/project/${jobPart()}/generate`, { method: "POST" }); let total = PROJECT.shots.length; await readSSE(r, ev => { if (ev.type === "start") { total = ev.total || total; logLine(`${ev.total} to draw, ${ev.already || 0} already done.`); } - else if (ev.type === "info") { logLine(ev.message || "", "warn"); } else if (ev.type === "shot") { const pct = Math.round((ev.done / Math.max(1, ev.total)) * 100); $("#gen-fill").style.width = pct + "%"; - $("#gen-label").textContent = `${pct}% · ${ev.done}/${ev.total}` + - (ev.backend_used ? ` · ${ev.backend_used}` : ""); + $("#gen-label").textContent = `${pct}% · ${ev.done}/${ev.total}`; const failed = ev.status === "error"; - const note = ev.note && ev.note !== "error" ? ` (${ev.note})` : ""; - logLine(`Shot ${ev.id}: ${failed ? "FAILED — " + (ev.error || "") : (ev.backend_used || "done") + note}`, - failed ? "err" : (ev.note && ev.note !== "error" ? "warn" : "")); + logLine(`Shot ${ev.id}: ${failed ? "FAILED — " + (ev.error || "") : "done"}`, + failed ? "err" : ""); const el = $(`#board .shot[data-id="${ev.id}"]`); const rec = findShot(ev.id); if (el && rec) { - rec.image_path = `images/${ev.id}.jpg`; rec.cache_bust = ev.cache_bust; - rec.backend_used = ev.backend_used; rec.status = ev.status; - const used = ev.backend_used || rec.backend; - $(".thumb", el).innerHTML = - `` + - `${fmtMs(rec.start_ms)}–${fmtMs(rec.end_ms)}` + - `${used === "stablediffusion" ? "SD" : used}`; + rec.image_path = failed ? null : `images/${ev.id}.jpg`; + if (!failed) { + $(".thumb", el).innerHTML = + `` + + `${fmtMs(rec.start_ms)}–${fmtMs(rec.end_ms)}`; + } else { + $(".thumb", el).innerHTML = + `generation failed` + + `${fmtMs(rec.start_ms)}–${fmtMs(rec.end_ms)}`; + } } + } else if (ev.type === "error") { + throw new Error(ev.message || "Generation stopped."); } else if (ev.type === "complete") { $("#gen-fill").style.width = "100%"; - $("#gen-label").textContent = "all shots drawn"; - $("#btn-to-assemble").disabled = false; - logLine("All shots drawn. Ready to stitch.", "ok"); - completed = true; + const failed = Number(ev.failed || 0); + if (failed) { + $("#gen-label").textContent = `${failed} shot${failed === 1 ? "" : "s"} need attention`; + $("#gen-err").textContent = `${failed} image${failed === 1 ? "" : "s"} failed. Regenerate the failed shot(s), then run Generate again.`; + $("#btn-to-assemble").disabled = true; + logLine(`${failed} shot(s) failed; video assembly is blocked until every image exists.`, "err"); + completed = false; + } else { + $("#gen-label").textContent = "all shots drawn"; + $("#btn-to-assemble").disabled = false; + logLine("All shots drawn. Ready to stitch.", "ok"); + completed = true; + } } }); } catch (e) { @@ -614,7 +598,7 @@ async function runAssemble() { $("#asm-err").textContent = ""; hide("results"); logLine("Stitching: rendering Ken Burns clips…"); try { - const r = await fetch(`/api/project/${PROJECT.job}/assemble`, { method: "POST" }); + const r = await fetch(`/api/project/${jobPart()}/assemble`, { method: "POST" }); await readSSE(r, ev => { if (ev.type === "start") { $("#asm-label").textContent = `rendering ${ev.total} clips…`; logLine(`${ev.total} clips to render.`); } else if (ev.type === "clip") { @@ -640,30 +624,28 @@ async function runAssemble() { } function showResults(ev) { - const job = PROJECT.job; + const job = jobPart(); + const encFile = name => String(name || "").split("/").map(encodeURIComponent).join("/"); const links = []; - links.push(`⬇ Download MP4`); + links.push(`⬇ Download MP4`); if (ev.timestamps_file) - links.push(`YouTube chapters .txt`); + links.push(`YouTube chapters .txt`); if (ev.drive_file) - links.push(`Google Drive chapter page`); + links.push(`Google Drive chapter page`); if (ev.subtitle_file) - links.push(`Subtitles (.srt)${ev.subtitle_mode === "burn" ? " · also burned in" : ""}`); + links.push(`Subtitles (.srt)${ev.subtitle_mode === "burn" ? " · also burned in" : ""}`); $("#dl-links").innerHTML = links.join(""); - $("#video-preview").src = `/image/${job}/${ev.video}`; - - // scene breakdown table (like Parroty's results table) - const rows = PROJECT.shots.map((s, i) => { - const used = s.backend_used || s.backend; - const srcLabel = used === "stablediffusion" ? "Stable Diffusion" - : used === "copilot" ? "Copilot" : "placeholder"; - const ok = s.status !== "error"; + $("#video-preview").src = `/image/${job}/${encFile(ev.video)}`; + + const rows = PROJECT.shots.map((shot, i) => { + const ok = shot.status !== "error"; + const title = escapeHtml(shot.chapter_title); + const status = ok ? "✓ done" : `✕ ${escapeHtml(shot.error || "error")}`; return ` ${i + 1} - ${s.chapter_title}${s.is_chapter_start ? ' ch start' : ""} - ${fmtMs(s.start_ms)}–${fmtMs(s.end_ms)} - ${srcLabel} - ${ok ? "✓ done" : "✕ " + (s.error || "error")} + ${title}${shot.is_chapter_start ? ' ch start' : ""} + ${fmtMs(shot.start_ms)}–${fmtMs(shot.end_ms)} + ${status} `; }).join(""); $("#restable-body").innerHTML = rows; @@ -683,8 +665,8 @@ async function loadResume() { : (p.done >= p.shots && p.shots) ? "all images ready" : `${p.done}/${p.shots} images drawn`; const when = fmtWhen(p.modified); - return `
-
${p.title || "Audiobook"} + return `
+
${escapeHtml(p.title || "Audiobook")} ${when ? when + " · " : ""}${fmtMs(p.total_ms)} · ${status}
@@ -711,7 +693,7 @@ async function loadResume() { async function resumeProject(job) { try { - const p = await (await fetch(`/api/project/${job}`)).json(); + const p = await (await fetch(`/api/project/${encodeURIComponent(job)}`)).json(); if (!p || p.error) throw new Error("Couldn't load that session."); PROJECT = p; renderBoard(); @@ -729,25 +711,6 @@ async function resumeProject(job) { } $("#resume-dismiss").addEventListener("click", () => hide("resume-bar")); -$("#btn-test-copilot")?.addEventListener("click", async () => { - const btn = $("#btn-test-copilot"), out = $("#copilot-test-result"); - const orig = btn.textContent; - btn.disabled = true; btn.textContent = "Testing… (a few seconds)"; - out.textContent = ""; out.className = "ctest-result"; - try { - const r = await fetch("/api/copilot_test", { method: "POST" }); - const d = await r.json(); - out.textContent = (d.ok ? "✓ " : "✗ ") + d.detail; - out.className = "ctest-result " + (d.ok ? "ok" : "warn"); - toast(d.detail, d.ok ? "" : "warn"); - if (!d.ok) $("#copilot-help").classList.remove("hidden"); - } catch { - out.textContent = "✗ Test couldn't run."; out.className = "ctest-result warn"; - } - btn.disabled = false; btn.textContent = orig; -}); -$("#copilot-help-toggle")?.addEventListener("click", () => - $("#copilot-help").classList.toggle("hidden")); $("#resume-clear")?.addEventListener("click", async () => { if (!confirm("Clear ALL recent sessions? This permanently deletes their images and any built videos from disk.")) return; try { @@ -757,3 +720,42 @@ $("#resume-clear")?.addEventListener("click", async () => { } catch { toast("Couldn't clear sessions.", "warn"); } }); loadResume(); + + +/* ── SeeStory Windows desktop lifecycle + system monitor ------------------ */ +(() => { + const heartbeat = () => fetch("/api/desktop/heartbeat", { + method: "POST", cache: "no-store", keepalive: true, + }).catch(() => {}); + heartbeat(); + window.setInterval(heartbeat, 2500); + + const pct = value => value == null ? "—" : `${Math.round(value)}%`; + const num = (value, digits = 1) => value == null ? "—" : Number(value).toFixed(digits); + + async function refreshSystemMonitor() { + const panel = document.getElementById("system-monitor"); + if (!panel) return; + try { + const data = await (await fetch("/api/system", { cache: "no-store" })).json(); + const gpu = data.gpu || {}; + document.getElementById("sm-cpu").textContent = pct(data.cpu_percent); + document.getElementById("sm-ram").textContent = + data.ram_used_gb == null ? "—" : `${num(data.ram_used_gb)}/${num(data.ram_total_gb)} GB`; + document.getElementById("sm-gpu").textContent = gpu.available ? pct(gpu.util) : "not detected"; + document.getElementById("sm-vram").textContent = gpu.available + ? `${Math.round(gpu.memory_used_mb)}/${Math.round(gpu.memory_total_mb)} MB` + : "—"; + document.getElementById("sm-temp").textContent = gpu.available + ? `${Math.round(gpu.temperature_c)}°C` + : "—"; + panel.classList.toggle("gpu-active", Boolean(gpu.available && gpu.util >= 5)); + panel.classList.remove("offline"); + } catch { + panel.classList.add("offline"); + } + } + + refreshSystemMonitor(); + window.setInterval(refreshSystemMonitor, 2500); +})(); diff --git a/app/static/seestory-icon.svg b/app/static/seestory-icon.svg new file mode 100644 index 0000000..8ccc562 --- /dev/null +++ b/app/static/seestory-icon.svg @@ -0,0 +1,58 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/app/static/style.css b/app/static/style.css index 9c5a47a..3552a4b 100644 --- a/app/static/style.css +++ b/app/static/style.css @@ -58,15 +58,6 @@ a { color: var(--beam); } font-family: var(--serif); font-style: italic; font-size: 1.1rem; color: var(--screen-soft); margin: 12px 0 0; } -.badges { display: flex; gap: 8px; justify-content: center; flex-wrap: wrap; margin-top: 18px; } -.badge { - font-family: var(--mono); font-size: .7rem; text-transform: uppercase; - letter-spacing: .05em; padding: 4px 10px; border-radius: 20px; - border: 1px solid var(--rule); color: var(--screen-soft); background: var(--panel); -} -.badge.on { color: var(--ok); border-color: rgba(121,178,119,.5); } -.badge.off { color: var(--screen-soft); opacity: .6; } -.badge.warn { color: var(--warn); border-color: rgba(221,154,78,.5); } /* ---- Steps ------------------------------------------------------------- */ main { max-width: 1080px; margin: 0 auto; padding: 8px 28px 90px; } @@ -109,12 +100,6 @@ main { max-width: 1080px; margin: 0 auto; padding: 8px 28px 90px; } font-family: var(--sans); font-size: .92rem; padding: 9px 11px; border: 1px solid var(--rule); border-radius: 6px; background: var(--panel-2); color: var(--screen); } -.mode-cards { display: flex; gap: 10px; } -.mode-card { flex: 1; border: 1.5px solid var(--rule); border-radius: 7px; padding: 12px; - cursor: pointer; background: var(--panel); text-align: center; } -.mode-card.sel { border-color: var(--beam); box-shadow: 0 0 0 1px var(--beam); } -.mode-card .mc-t { font-family: var(--serif); font-weight: 600; } -.mode-card .mc-s { font-size: .74rem; color: var(--screen-soft); margin-top: 3px; } /* ---- Buttons ----------------------------------------------------------- */ button { font-family: var(--sans); cursor: pointer; } @@ -142,7 +127,6 @@ button { font-family: var(--sans); cursor: pointer; } border-image: repeating-linear-gradient(90deg, var(--rule) 0 8px, transparent 8px 16px) 6; } .shot.chapter-start { box-shadow: inset 3px 0 0 var(--beam); } -.shot.is-copilot { border-color: rgba(232,162,60,.5); } .thumb { position: relative; aspect-ratio: 16/9; background: var(--room-deep); display: grid; place-items: center; overflow: hidden; } .thumb img { width: 100%; height: 100%; object-fit: cover; display: block; } @@ -152,11 +136,6 @@ button { font-family: var(--sans); cursor: pointer; } @keyframes sp { to { transform: rotate(360deg); } } .thumb .ttime { position: absolute; left: 8px; bottom: 8px; font-family: var(--mono); font-size: .68rem; background: rgba(0,0,0,.55); color: var(--screen); padding: 2px 7px; border-radius: 4px; } -.thumb .tbk { position: absolute; right: 8px; top: 8px; font-family: var(--mono); - font-size: .62rem; text-transform: uppercase; letter-spacing: .05em; padding: 2px 7px; - border-radius: 4px; background: rgba(0,0,0,.55); } -.tbk.copilot { color: var(--beam); } .tbk.stablediffusion { color: var(--glow); } -.tbk.placeholder { color: var(--screen-soft); } .shot-body { padding: 12px 13px 14px; display: flex; flex-direction: column; gap: 10px; } .shot-chap { font-family: var(--mono); font-size: .68rem; color: var(--screen-soft); @@ -167,10 +146,6 @@ button { font-family: var(--sans); cursor: pointer; } .shot-prompt:focus { outline: none; border-color: var(--beam); } .row { display: flex; gap: 8px; align-items: center; flex-wrap: wrap; } -.seg { display: flex; border: 1px solid var(--rule); border-radius: 6px; overflow: hidden; } -.seg button { background: var(--panel-2); border: none; color: var(--screen-soft); - padding: 5px 9px; font-family: var(--mono); font-size: .68rem; } -.seg button.on { background: var(--beam); color: #1a1206; } .motion { border-top: 1px dashed var(--rule); padding-top: 10px; display: none; flex-direction: column; gap: 9px; } .motion.open { display: flex; } @@ -269,9 +244,6 @@ footer { max-width: 1080px; margin: 0 auto; padding: 28px; text-align: center; letter-spacing: .06em; color: var(--beam); border: 1px solid var(--beam); border-radius: 6px; padding: 1px 5px; margin-left: 6px; opacity: .8; } -/* standalone source badges (sample meta + results table) */ -.sample-meta .tbk, .restable .tbk { font-family: var(--mono); font-size: 10px; - text-transform: uppercase; letter-spacing: .04em; position: static; } /* ── resume / restore bar ───────────────────────────────────────────────── */ .resume { background: var(--panel); border: 1px solid var(--beam-deep); @@ -335,17 +307,12 @@ footer { max-width: 1080px; margin: 0 auto; padding: 28px; text-align: center; .gm-head .gm-master { margin-bottom: 0; } #btn-apply-motion-all { white-space: nowrap; } -/* sign-in reminder under the badges */ -.signin-note { margin: 8px 0 0; font-size: .82rem; color: var(--beam); - background: rgba(214,178,106,.08); border: 1px solid rgba(214,178,106,.3); - border-radius: 8px; padding: 7px 11px; max-width: 760px; } -.signin-note code { font-family: var(--mono); font-size: .92em; } -/* transient toast (e.g. why a single regenerate fell back) */ +/* transient toast for local errors and confirmations */ .toast { position: fixed; bottom: 22px; left: 50%; transform: translateX(-50%) translateY(18px); background: var(--panel-2); color: var(--screen); - border: 1px solid var(--line); border-radius: 10px; padding: 11px 16px; + border: 1px solid var(--rule); border-radius: 10px; padding: 11px 16px; max-width: 560px; font-size: .85rem; line-height: 1.35; box-shadow: 0 10px 34px rgba(0,0,0,.55); opacity: 0; transition: opacity .35s, transform .35s; z-index: 1000; } @@ -356,28 +323,30 @@ footer { max-width: 1080px; margin: 0 auto; padding: 28px; text-align: center; .resume-head-actions { display: flex; gap: 14px; align-items: center; } .link.danger:hover { color: #d98a7a; } .ri-actions { display: flex; align-items: center; gap: 8px; } -.ri-del { background: transparent; border: 1px solid var(--line); +.ri-del { background: transparent; border: 1px solid var(--rule); color: var(--screen-soft); border-radius: 6px; width: 26px; height: 26px; cursor: pointer; font-size: .78rem; line-height: 1; padding: 0; } .ri-del:hover { border-color: #d98a7a; color: #d98a7a; } -/* Copilot test + troubleshooting */ -.copilot-tools { display: flex; align-items: center; gap: 12px; margin: 8px 0 0; - flex-wrap: wrap; } -.ctest-result { font-size: .82rem; color: var(--screen-soft); } -.ctest-result.ok { color: var(--glow); } -.ctest-result.warn { color: var(--beam); } -.copilot-help { margin: 8px 0 0; font-size: .82rem; color: var(--screen-soft); - background: rgba(255,255,255,.03); border: 1px solid var(--line); - border-radius: 8px; padding: 8px 12px 8px 4px; max-width: 760px; } -.copilot-help ol { margin: 6px 0 0; padding-left: 22px; } -.copilot-help li { margin: 4px 0; line-height: 1.4; } -.copilot-help code { font-family: var(--mono); } -.copilot-help a { color: var(--beam); } - -/* inline help icon with hover tooltip */ -.help { display: inline-block; width: 16px; height: 16px; line-height: 16px; - text-align: center; border-radius: 50%; font-size: 11px; cursor: help; - color: var(--screen-soft); border: 1px solid var(--line); margin-left: 4px; - white-space: pre-line; } -.help:hover, .help:focus { color: var(--beam); border-color: var(--beam); outline: none; } + + +/* SeeStory desktop system monitor ----------------------------------------- */ +.system-monitor { + position: fixed; left: 14px; bottom: 14px; z-index: 80; + width: 218px; padding: 10px 11px 9px; + border: 1px solid rgba(232,162,60,.38); border-radius: 12px; + background: rgba(14,11,9,.90); box-shadow: 0 12px 40px rgba(0,0,0,.44); + backdrop-filter: blur(12px); color: var(--cream, #ece0c8); + font: 11px/1.35 "Spline Sans Mono", ui-monospace, monospace; + user-select: none; transition: opacity .2s, border-color .2s; +} +.system-monitor.offline { opacity: .55; } +.system-monitor.gpu-active { border-color: rgba(91,210,188,.70); } +.sm-head { display: flex; align-items: center; gap: 7px; margin-bottom: 7px; text-transform: uppercase; letter-spacing: .08em; } +.sm-dot { width: 7px; height: 7px; border-radius: 50%; background: #e8a23c; box-shadow: 0 0 9px rgba(232,162,60,.9); } +.gpu-active .sm-dot { background: #5bd2bc; box-shadow: 0 0 9px rgba(91,210,188,.9); } +.sm-priority { margin-left: auto; color: #cfaa73; font-size: 9px; letter-spacing: .04em; } +.sm-grid { display: grid; grid-template-columns: 66px 1fr; gap: 3px 9px; } +.sm-grid span { color: #9e8e7a; } +.sm-grid strong { color: #f3dfb8; font-weight: 600; text-align: right; white-space: nowrap; } +@media (max-width: 760px) { .system-monitor { display: none; } } diff --git a/app/templates/index.html b/app/templates/index.html index 454c1a6..9d49992 100644 --- a/app/templates/index.html +++ b/app/templates/index.html @@ -8,7 +8,7 @@ - +
@@ -36,36 +36,6 @@

SeeStory

Bring your narrated audiobook to life.

-
- ffmpeg {{ 'ready' if ffmpeg_ok else 'missing' }} - stable diffusion {{ 'ready' if probe.stablediffusion else 'not installed' }} - {% if probe.stablediffusion %}{{ 'GPU' if probe.cuda else 'CPU only' }}{% endif %} - {% if probe.copilot and probe.copilot_signed_in %}copilot signed in - {% elif probe.copilot %}copilot — sign in (run login_copilot.bat) - {% else %}copilot not set up{% endif %} - placeholder always on -
- {% if probe.copilot and not probe.copilot_signed_in %} - - {% endif %} - {% if probe.copilot %} -
- - - -
- - {% endif %}
@@ -113,23 +83,8 @@

SeeStory

ii

How it should look

-

Stable Diffusion draws every page on your own GPU (free, unlimited). Copilot is saved for the - standout moments — gorgeous, but rate-limited, so it's capped to protect your account.

- -
- -
-
-
Both
SD every page · Copilot for the big moments
-
-
-
Stable Diffusion only
no account, fully local
-
-
-
Copilot only
premium, capped & spaced
-
-
-
+

SeeStory generates every image locally on your NVIDIA GPU. Pick the visual style and + how often the scene changes; the image model, quality settings, and anatomy cleanup are handled automatically.

@@ -144,21 +99,13 @@

SeeStory

'ink': 'Ink & watercolor'} %} {% for s in styles %}{% endfor %} -

Pick Photorealistic for real-looking photos instead of a drawing. The first time you use it, it downloads a photo-quality model (~6 GB), then it's cached.

+

Pick Photorealistic for a real-camera look. That optional model must be selected and GPU-tested during install_all.bat.

Describe the visual art style only — medium, palette, mood. Don't put the plot, genre, author, or the word "book" here; words like "book" or a title make the pictures come out with garbled text on them.

-
- - -
@@ -168,18 +115,8 @@

SeeStory

-
- - -
-
- - -
-
- -
- +
+ +
Leave blank and SeeStory will auto-generate approximate subtitles from the ebook text (good for Parroty's steady narration, but not frame-perfect).
+
- Draws one random page with Stable Diffusion so you can preview the style. First run loads the model (can take a minute). + Draws one random page locally using the model installed and GPU-tested by install_all.bat.