Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 39 additions & 5 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,12 @@ C# .NET 10 console app for scraping and converting audiobooks (M4B/MP3) from mul
IScraperStrategy (Scraper/Abstractions/)
└── BaseScraperStrategy (Scraper/Base/)
├── TokybookStrategy (Scraper/Strategies/) — tokybook.com
├── ZAudiobooksStrategy (Scraper/Strategies/) — zaudiobooks / freeaudiobooks.top
├── GoldenAudiobookStrategy (Scraper/Strategies/) — all sites with `<source type="audio/mpeg">` structure:
├── DropboxTracksStrategy (Scraper/Strategies/) — sites with JS `tracks = [{ chapter_link_dropbox }]` structure:
│ zaudiobooks.com, freeaudiobooks.top
├── AudioSourceTagStrategy (Scraper/Strategies/) — sites with `<source type="audio/mpeg">` or `<a href="*.mp3">` structure:
│ goldenaudiobook.net, fulllengthaudiobooks.net, bigaudiobooks.net,
│ findaudiobook.com, bookaudiobook.net, hotaudiobooks.com, audiozaic.com
│ findaudiobook.com, bookaudiobook.net, hotaudiobooks.com, audiozaic.com,
│ appaudiobooks.com
├── PlaylistAudiobookStrategy (Scraper/Strategies/) — all sites with `data-playlist` JSON attribute:
│ hdaudiobooks.net
└── AudioAzStrategy (Scraper/Strategies/) — Next.js site with tracks JSON in streaming data:
Expand All @@ -44,16 +46,47 @@ Downloads and conversions run decoupled via `Channel<T>` + `SemaphoreSlim`:
### Track Types

- `SegmentedTrackData` — for HLS streams (`.m3u8` → `.ts` segments → merge via FFmpeg concat)
- `DirectFileTrackData` — for direct MP3/audio downloads (zaudiobooks, goldenaudiobook); conversion is skipped when source file is already in the target format
- `DirectFileTrackData` — for direct MP3/audio downloads (zaudiobooks, goldenaudiobook); when the source is already in the target format, a copy-conversion runs to embed metadata without re-encoding

### Metadata Pipeline

`BaseScraperStrategy` provides shared metadata infrastructure used by all strategies.

For all non-Tokybook strategies, metadata is collected in two stages before downloading:

**Stage 1 — MP3 tag enrichment** (`EnrichFromFirstTrackTagsAsync`):
- Runs `ffprobe` on the first chapter URL to read existing ID3 tags without downloading the file
- Maps: `artist` → `Author`, `date` → `Year`, `comment` → `Description` (skips chapter references like "Chapter 1")
- Only fills empty fields — never overwrites

**Stage 2 — HTML extraction** (`ExtractCommonMetadata`):
- Only fills fields still empty after Stage 1
- `og:image` → `CoverArtUrl`
- `og:description` → `Description`
- `<script type="application/ld+json">` with `@type:"Audiobook"` → all fields (AudioAZ)
- `ld+json` `headline` field → author via `ExtractAuthorFromHeadline()` (WordPress/Yoast sites)
- `<link rel="preload" as="image">` → `CoverArtUrl` fallback (fulllengthaudiobooks, appaudiobooks)
- H1 title → author as last resort

Other shared helpers:
- `DownloadCoverArtAsync(url, folder)` — downloads cover once to `_cover.{ext}`, returns temp path
- `BuildMetadataParams(bookMetadata, trackData, hasCoverArt)` — returns FFmpeg `-metadata` flags (title, album, artist, album_artist, track, genre, comment, publisher, date, cover art)
- Cover art is passed to FFmpeg as a second input (`-map 0:a -map 1:v -c:v copy -disposition:v attached_pic`)
- Cover art temp file is always cleaned up via `finally` after the conversion pipeline completes

**Tokybook** gets richer metadata directly from the `post-details` API response (`authors`, `narrators`, `coverImage`, `description`, `publisher`) — no HTML scraping or ffprobe needed.

### Data Model

```
AudiobookMetadata (abstract)
│ Title, FolderPath
│ Author, Narrator, CoverArtUrl, Description, Publisher, Year ← populated by ffprobe tags, HTML, or API response
├── SimpleAudiobookMetadata — ChapterUrls: List<string>
└── StreamingAudiobookMetadata — Tracks: List<TrackInfo>, StreamToken, AudioBookId

TrackData (abstract)
│ TrackTitle, SanitizedTitle, TrackNumber, TotalTracks
├── SegmentedTrackData — TempFolder, FolderPath, TsSegments: List<string>
└── DirectFileTrackData — FilePath, FolderPath

Expand Down Expand Up @@ -114,7 +147,8 @@ Registration in `Program.cs` → `ConfigureServices()` and `ScraperServiceExtens
1. Create a new class in `TokyBay/Scraper/Strategies/` extending `BaseScraperStrategy`
2. Implement `CanHandle(string url)` — URL-based detection
3. Implement `DownloadBookAsync(string url)` — fetch metadata, then call `ProcessTracksInParallelAsync` or `ProcessDirectFilesInParallelAsync`
4. Register in `ScraperServiceExtensions.cs`:
4. In the metadata fetch method, call `ExtractCommonMetadata(html, metadata)` after building the `SimpleAudiobookMetadata` object — this fills cover art, author, description automatically from og-tags and ld+json
5. Register in `ScraperServiceExtensions.cs`:
```csharp
services.AddTransient<IScraperStrategy, NewStrategy>();
```
Expand Down
24 changes: 21 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# TokyBay

Search & download audiobooks from multiple sites and convert them automatically to the audiobook-friendly M4B format or good old MP3.
Search, download, and tag audiobooks from multiple sites automatically converted to M4B or MP3, with cover art, author, narrator, and chapter titles embedded directly into every file.

> [!Important]
> Tokybook.com is currently undergoing a major platform overhaul and the website is temporarily closed. However, **TokyBay continues to work** — search and downloads still function via the Tokybook API.
Expand Down Expand Up @@ -72,6 +72,7 @@ That's it. TokyBay will guide you through the rest.

| Site | Status |
|------|--------|
| [appaudiobooks.com](https://appaudiobooks.com/) | 🟢 Working |
| [audioaz.com](https://audioaz.com) | 🟢 Working |
| [audiozaic.com](https://audiozaic.com) | 🟢 Working |
| [bigaudiobooks.net](https://bigaudiobooks.net) | 🟢 Working |
Expand All @@ -91,8 +92,25 @@ That's it. TokyBay will guide you through the rest.
2. **Direct URL download**: Download any audiobook directly by URL on supported sites
3. **M4B conversion**: Automatically convert to the M4B audiobook format after download
4. **MP3 conversion**: Automatically convert to MP3 format after download
5. **Multi-site support**: Works with Tokybook and many other sites
6. **Settings**: Persistent in-app settings — download path, conversion preferences
5. **Metadata tagging**: Embeds rich metadata into every converted file — cover art, author, narrator, chapter title, track number, genre, description, and publisher
6. **Multi-site support**: Works with Tokybook and many other sites
7. **Settings**: Persistent in-app settings — download path, conversion preferences

### Embedded Metadata

Every downloaded chapter is tagged automatically so your audiobook player displays the right information:

| Tag | Source |
|-----|--------|
| Cover art | Fetched from the book page and embedded as attached picture |
| Author | Read from original MP3 tags first, then extracted from the site's structured data or page title |
| Narrator | Extracted where available (Tokybook and AudioAZ carry this natively) |
| Year | Read from original MP3 tags (publication year is often embedded in the source file) |
| Chapter title | File name or track title from the playlist |
| Track number | Chapter position and total count (e.g. `3/24`) |
| Genre | Always set to `Audiobook` |
| Description | Read from original MP3 tags first, then extracted from book synopsis on the page |
| Publisher | Where available (Tokybook carries this natively) |

## Usage

Expand Down
6 changes: 6 additions & 0 deletions TokyBay/Models/AudiobookMetadata.cs
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,11 @@ public abstract class AudiobookMetadata
{
public string Title { get; set; } = string.Empty;
public string FolderPath { get; set; } = string.Empty;
public string Author { get; set; } = string.Empty;
public string Narrator { get; set; } = string.Empty;
public string CoverArtUrl { get; set; } = string.Empty;
public string Description { get; set; } = string.Empty;
public string Publisher { get; set; } = string.Empty;
public string Year { get; set; } = string.Empty;
}
}
Loading