Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,14 @@ jobs:
git clone "https://x-access-token:${{ github.token }}@github.com/${{ github.repository }}.git" .
git checkout ${{ github.event.pull_request.merge_commit_sha }}

- name: Install .NET 10 SDK
run: |
wget https://dot.net/v1/dotnet-install.sh -O dotnet-install.sh
chmod +x dotnet-install.sh
./dotnet-install.sh --channel 10.0
echo "$HOME/.dotnet" >> $GITHUB_PATH
echo "DOTNET_ROOT=$HOME/.dotnet" >> $GITHUB_ENV

- name: Extract version from csproj
id: version
run: |
Expand Down
147 changes: 114 additions & 33 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,18 @@
# TokyBay

C# .NET 9 Console App zum Scrapen und Konvertieren von Audiobüchern (M4B/MP3) von mehreren Websites.
C# .NET 10 console app for scraping and converting audiobooks (M4B/MP3) from multiple websites.

## Stack

- **.NET 9**, C# mit Nullable-Referenztypen und Primary Constructors
- **Spectre.Console** — alle Konsolenausgaben (Markup mit `[green]...[/]` etc., Status-Spinner)
- **Xabe.FFmpeg** — Audio-Konvertierung und Segment-Merging; Binaries via `Xabe.FFmpeg.Downloader`
- **Newtonsoft.Json** — JSON-Parsing der API-Antworten
- **Microsoft.Extensions.DependencyInjection** — DI-Container
- **.NET 10**, C# with nullable reference types and primary constructors
- **Spectre.Console** — all console output (markup with `[green]...[/]` etc., status spinners, tables, figlet)
- **Xabe.FFmpeg** — audio conversion and segment merging; binaries via `Xabe.FFmpeg.Downloader`
- **Newtonsoft.Json** — JSON parsing of API responses
- **Microsoft.Extensions.DependencyInjection** — DI container
- **Microsoft.Extensions.Configuration** — configuration via `appsettings.json` with binder
- **Microsoft.Extensions.Http** — `IHttpClientFactory` for typed HttpClients

## Architektur
## Architecture

### Strategy Pattern (Scraper)

Expand All @@ -19,43 +21,109 @@ IScraperStrategy (Scraper/Abstractions/)
└── BaseScraperStrategy (Scraper/Base/)
├── TokybookStrategy (Scraper/Strategies/) — tokybook.com
├── ZAudiobooksStrategy (Scraper/Strategies/) — zaudiobooks / freeaudiobooks.top
├── GoldenAudiobookStrategy (Scraper/Strategies/) — alle Sites mit `<source type="audio/mpeg">` Struktur:
├── GoldenAudiobookStrategy (Scraper/Strategies/) — all sites with `<source type="audio/mpeg">` structure:
│ goldenaudiobook.net, fulllengthaudiobooks.net, bigaudiobooks.net,
│ findaudiobook.com, bookaudiobook.net, hotaudiobooks.com, audiozaic.com
└── PlaylistAudiobookStrategy (Scraper/Strategies/) — alle Sites mit `data-playlist` JSON-Attribut:
hdaudiobooks.net
├── PlaylistAudiobookStrategy (Scraper/Strategies/) — all sites with `data-playlist` JSON attribute:
│ hdaudiobooks.net
└── AudioAzStrategy (Scraper/Strategies/) — Next.js site with tracks JSON in streaming data:
audioaz.com
```

- `ScraperFactory` wählt per `CanHandle(url)` die passende Strategie aus
- `ScraperConfig` steuert Parallelismus-Parameter (Defaults: 3 parallele Downloads, 2 Konvertierungen, 5 Segmente/Track)
- `ScraperFactory` selects the matching strategy via `CanHandle(url)`
- `ScraperConfig` controls parallelism parameters (defaults: 3 parallel downloads, 2 conversions, 5 segments/track)
- Runtime overrides in `Program.cs`: MaxParallelDownloads=5, MaxParallelConversions=3, MaxSegmentsPerTrack=8

### Download-Pipeline
### Download Pipeline

Downloads und Konvertierungen laufen entkoppelt über `Channel<T>` + `SemaphoreSlim`:
1. Download-Tasks schreiben fertige Tracks in einen bounded Channel
2. Konvertierungs-Tasks lesen aus dem Channel und rufen FFmpeg auf
3. Channel wird nach `Task.WhenAll(downloadTasks)` geschlossen
Downloads and conversions run decoupled via `Channel<T>` + `SemaphoreSlim`:
1. Download tasks write completed tracks into a bounded channel
2. Conversion tasks read from the channel and invoke FFmpeg
3. Channel is closed after `Task.WhenAll(downloadTasks)`

### Track-Typen
### Track Types

- `SegmentedTrackData` — für HLS-Streams (`.m3u8` → `.ts`-Segmente → merge via FFmpeg concat)
- `DirectFileTrackData` — für direkte MP3/Audio-Downloads (zaudiobooks, goldenaudiobook); Konvertierung wird übersprungen wenn Quelldatei bereits im Zielformat vorliegt
- `SegmentedTrackData` — for HLS streams (`.m3u8` → `.ts` segments → merge via FFmpeg concat)
- `DirectFileTrackData` — for direct MP3/audio downloads (zaudiobooks, goldenaudiobook); conversion is skipped when source file is already in the target format

## Neue Website hinzufügen
### Data Model

1. Neue Klasse in `TokyBay/Scraper/Strategies/` anlegen, die von `BaseScraperStrategy` erbt
2. `CanHandle(string url)` implementieren — URL-basierte Erkennung
3. `DownloadBookAsync(string url)` implementieren — Metadata fetch, dann `ProcessTracksInParallelAsync`
4. In `ScraperServiceExtensions.cs` registrieren:
```
AudiobookMetadata (abstract)
├── SimpleAudiobookMetadata — ChapterUrls: List<string>
└── StreamingAudiobookMetadata — Tracks: List<TrackInfo>, StreamToken, AudioBookId

TrackData (abstract)
├── SegmentedTrackData — TempFolder, FolderPath, TsSegments: List<string>
└── DirectFileTrackData — FilePath, FolderPath

TrackInfo — Src, TrackTitle
UserSettings — DownloadPath, FFmpegDirectory, ConvertToMp3, ConvertToM4b
```

### Services

- `HttpService` (`IHttpService`) — GET/POST via `HttpClient` (10 min timeout, DI via `IHttpClientFactory`)
- `DownloadService` — orchestrates strategy selection and execution, provides supported domains
- `SettingsService` (`ISettingsService`) — loads/saves `UserSettings` in `appsettings.json`, auto-downloads FFmpeg
- `IpifyService` (`IIpifyService`) — fetches IP address via `api.ipify.org` (required for Tokybook API)
- `PageService` (`IPageService`) — Spectre.Console UI wrapper with figlet banner (`Bulbhead.flf`), ESC cancellation

### Pages (UI Flow)

```
Application.RunAsync(args)
→ InitializeAsync() — load settings, ensure FFmpeg
→ Parse CLI args (-d / --directory)
→ MainPage.ShowAsync() — main menu loop
├── "Search book on Tokybook.com" → SearchTokybookPage (API: /api/v1/search, pagination)
├── "Download from URL" → DownloadPage (shows supported domains, prompts for URL)
├── "Settings" → SettingsPage (download path, FFmpeg path, MP3/M4B toggles)
└── "Exit"
```

### Dependency Injection

Registration in `Program.cs` → `ConfigureServices()` and `ScraperServiceExtensions.AddScraperServices()`:

- Singleton: `IConfiguration`, `IAnsiConsole`, `EscapeCancellableConsole`, `IIpifyService`, `IPageService`, `ISettingsService`, `ScraperFactory`, `DownloadService`
- Typed HttpClient: `IHttpService` → `HttpService`
- Transient: all `IScraperStrategy` implementations

## Configuration

### appsettings.json

```json
{
"UserSettings": {
"DownloadPath": "",
"FFmpegDirectory": "",
"ConvertToMp3": true,
"ConvertToM4b": false
}
}
```

### CLI Arguments

- `-d` / `--directory` `<path>` — sets download directory

## Adding a New Website

1. Create a new class in `TokyBay/Scraper/Strategies/` extending `BaseScraperStrategy`
2. Implement `CanHandle(string url)` — URL-based detection
3. Implement `DownloadBookAsync(string url)` — fetch metadata, then call `ProcessTracksInParallelAsync` or `ProcessDirectFilesInParallelAsync`
4. Register in `ScraperServiceExtensions.cs`:
```csharp
services.AddTransient<IScraperStrategy, NeueStrategy>();
services.AddTransient<IScraperStrategy, NewStrategy>();
```

## Build & Run

```sh
dotnet build
dotnet run --project TokyBay -- -d "C:\Pfad\zum\Download"
dotnet run --project TokyBay -- -d "C:\Path\To\Downloads"
```

### Publish (Cross-Platform)
Expand All @@ -68,10 +136,23 @@ dotnet publish -c Release -r osx-x64 --self-contained
dotnet publish -c Release -r osx-arm64 --self-contained
```

## Konventionen
Publish profiles are in `TokyBay/Properties/PublishProfiles/` (WinX64, LinuxX64, LinuxArm64).
Flags: `PublishSingleFile=true`, `PublishTrimmed=false`, `SelfContained=true`.

## CI/CD (GitHub Actions)

Workflow: `.github/workflows/release.yml`

- **Trigger:** Pull request merged into `master`
- **Constraint:** Org `z00mable` only allows actions owned by the org — no third-party actions like `actions/checkout`, `actions/setup-dotnet`, `softprops/action-gh-release`. Use `git clone`/`run`-based alternatives instead (e.g. `dotnet-install.sh` for SDK setup, `gh release create` instead of `softprops/action-gh-release`).
- **Steps:** git clone → install .NET 10 SDK (via `dotnet-install.sh`) → extract version from `.csproj` → publish for 5 platforms (win-x64, linux-x64, linux-arm64, osx-x64, osx-arm64) → ZIP → git tag `v{version}` → GitHub Release via `gh release create`
- **Release notes:** From PR body

## Conventions

- Konsolenausgaben immer via `_console.MarkupLine(...)` (Spectre), nie `Console.WriteLine`
- Fehlermeldungen in `[red]`, Erfolg in `[green]`, Info in `[blue]`, Konvertierungen in `[cyan]`
- Dateinamen werden via `SanitizeName()` bereinigt (`[^A-Za-z0-9]+` → `_`)
- Retry-Logik: `RetryAsync<T>()` aus `BaseScraperStrategy` verwenden (exponentielles Delay)
- Temp-Verzeichnisse immer via `SafeDeleteDirectory()` aufräumen
- Console output always via `_console.MarkupLine(...)` (Spectre), never `Console.WriteLine`
- Error messages in `[red]`, success in `[green]`, info in `[blue]`, conversions in `[cyan]`, warnings in `[yellow]`, secondary in `[grey]`
- Filenames are sanitized via `SanitizeName()` (`[^A-Za-z0-9]+` → `_`)
- Retry logic: use `RetryAsync<T>()` from `BaseScraperStrategy` (exponential delay)
- Temp directories always cleaned up via `SafeDeleteDirectory()`
- ESC cancellation in UI prompts via `EscapeCancellableConsole`
29 changes: 15 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,21 +68,22 @@ That's it. TokyBay will guide you through the rest.

## Supported Sites

*Last checked: 2026-04-07*
*Last checked: 2026-04-08*

| Site | Status |
|------|--------|
| [tokybook.com](https://tokybook.com) | <font color="red">Website temporarily offline</font>, <font color="green">API operational</font> |
| [hdaudiobooks.net](https://hdaudiobooks.net) | <font color="green">Working</font> |
| [goldenaudiobook.net](https://goldenaudiobook.net) | <font color="green">Working</font> |
| [fulllengthaudiobooks.net](https://fulllengthaudiobooks.net) | <font color="green">Working</font> |
| [bigaudiobooks.net](https://bigaudiobooks.net) | <font color="green">Working</font> |
| [findaudiobook.com](https://findaudiobook.com) | <font color="green">Working</font> |
| [bookaudiobook.net](https://bookaudiobook.net) | <font color="green">Working</font> |
| [hotaudiobooks.com](https://hotaudiobooks.com) | <font color="green">Working</font> |
| [audiozaic.com](https://audiozaic.com) | <font color="green">Working</font> |
| [zaudiobooks.com](https://zaudiobooks.com) | <font color="red">Currently unreachable</font> |
| [freeaudiobooks.top](https://freeaudiobooks.top) | <font color="red">Currently unreachable</font> |
| [audioaz.com](https://audioaz.com) | 🟢 Working |
| [audiozaic.com](https://audiozaic.com) | 🟢 Working |
| [bigaudiobooks.net](https://bigaudiobooks.net) | 🟢 Working |
| [bookaudiobook.net](https://bookaudiobook.net) | 🟢 Working |
| [findaudiobook.com](https://findaudiobook.com) | 🟢 Working |
| [freeaudiobooks.top](https://freeaudiobooks.top) | 🔴 Currently unreachable |
| [fulllengthaudiobooks.net](https://fulllengthaudiobooks.net) | 🟢 Working |
| [goldenaudiobook.net](https://goldenaudiobook.net) | 🟢 Working |
| [hdaudiobooks.net](https://hdaudiobooks.net) | 🟢 Working |
| [hotaudiobooks.com](https://hotaudiobooks.com) | 🟢 Working |
| [tokybook.com](https://tokybook.com) | 🔴 Website temporarily offline, 🟢 API operational |
| [zaudiobooks.com](https://zaudiobooks.com) | 🔴 Currently unreachable |

## Features

Expand Down Expand Up @@ -116,7 +117,7 @@ That's it. TokyBay will guide you through the rest.
- Change the FFmpeg binary path

> [!Tip]
> By default, TokyBay downloads to the folder it is run from. You can change this inside the app's settings, or pass a custom path via `-d "C:\Users\User\Music"` when launching from the terminal.
> By default, TokyBay downloads to your **Music** folder on Windows (`C:\Users\<User>\Music`) or your **home directory** on Linux/macOS. You can change this inside the app's settings, or pass a custom path via `-d "C:\Path\To\Downloads"` when launching from the terminal.

---

Expand All @@ -127,7 +128,7 @@ That's it. TokyBay will guide you through the rest.

### Prerequisites

- [.NET 9 SDK](https://dotnet.microsoft.com/download/dotnet/9.0)
- [.NET 10 SDK](https://dotnet.microsoft.com/download/dotnet/10.0)

### Clone and run

Expand Down
4 changes: 2 additions & 2 deletions TokyBay/Properties/PublishProfiles/LinuxArm64.pubxml
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,10 @@
<PropertyGroup>
<Configuration>Release</Configuration>
<Platform>Any CPU</Platform>
<PublishDir>bin\Release\net9.0\publish\linux-arm64\</PublishDir>
<PublishDir>bin\Release\net10.0\publish\linux-arm64\</PublishDir>
<PublishProtocol>FileSystem</PublishProtocol>
<_TargetId>Folder</_TargetId>
<TargetFramework>net9.0</TargetFramework>
<TargetFramework>net10.0</TargetFramework>
<RuntimeIdentifier>linux-arm64</RuntimeIdentifier>
<SelfContained>true</SelfContained>
<PublishSingleFile>true</PublishSingleFile>
Expand Down
4 changes: 2 additions & 2 deletions TokyBay/Properties/PublishProfiles/LinuxX64.pubxml
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,10 @@
<PropertyGroup>
<Configuration>Release</Configuration>
<Platform>Any CPU</Platform>
<PublishDir>bin\Release\net9.0\publish\linux-x64\</PublishDir>
<PublishDir>bin\Release\net10.0\publish\linux-x64\</PublishDir>
<PublishProtocol>FileSystem</PublishProtocol>
<_TargetId>Folder</_TargetId>
<TargetFramework>net9.0</TargetFramework>
<TargetFramework>net10.0</TargetFramework>
<RuntimeIdentifier>linux-x64</RuntimeIdentifier>
<SelfContained>true</SelfContained>
<PublishSingleFile>true</PublishSingleFile>
Expand Down
16 changes: 16 additions & 0 deletions TokyBay/Properties/PublishProfiles/OsxArm64.pubxml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
<?xml version="1.0" encoding="utf-8"?>
<!-- https://go.microsoft.com/fwlink/?LinkID=208121. -->
<Project>
<PropertyGroup>
<Configuration>Release</Configuration>
<Platform>Any CPU</Platform>
<PublishDir>bin\Release\net10.0\publish\osx-arm64\</PublishDir>
<PublishProtocol>FileSystem</PublishProtocol>
<_TargetId>Folder</_TargetId>
<TargetFramework>net10.0</TargetFramework>
<RuntimeIdentifier>osx-arm64</RuntimeIdentifier>
<SelfContained>true</SelfContained>
<PublishSingleFile>true</PublishSingleFile>
<PublishTrimmed>false</PublishTrimmed>
</PropertyGroup>
</Project>
16 changes: 16 additions & 0 deletions TokyBay/Properties/PublishProfiles/OsxX64.pubxml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
<?xml version="1.0" encoding="utf-8"?>
<!-- https://go.microsoft.com/fwlink/?LinkID=208121. -->
<Project>
<PropertyGroup>
<Configuration>Release</Configuration>
<Platform>Any CPU</Platform>
<PublishDir>bin\Release\net10.0\publish\osx-x64\</PublishDir>
<PublishProtocol>FileSystem</PublishProtocol>
<_TargetId>Folder</_TargetId>
<TargetFramework>net10.0</TargetFramework>
<RuntimeIdentifier>osx-x64</RuntimeIdentifier>
<SelfContained>true</SelfContained>
<PublishSingleFile>true</PublishSingleFile>
<PublishTrimmed>false</PublishTrimmed>
</PropertyGroup>
</Project>
4 changes: 2 additions & 2 deletions TokyBay/Properties/PublishProfiles/WinX64.pubxml
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,10 @@
<PropertyGroup>
<Configuration>Release</Configuration>
<Platform>Any CPU</Platform>
<PublishDir>bin\Release\net9.0\publish\win-x64\</PublishDir>
<PublishDir>bin\Release\net10.0\publish\win-x64\</PublishDir>
<PublishProtocol>FileSystem</PublishProtocol>
<_TargetId>Folder</_TargetId>
<TargetFramework>net9.0</TargetFramework>
<TargetFramework>net10.0</TargetFramework>
<RuntimeIdentifier>win-x64</RuntimeIdentifier>
<SelfContained>true</SelfContained>
<PublishSingleFile>true</PublishSingleFile>
Expand Down
Loading