Skip to content

Latest commit

 

History

182 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Document Finder

Document Finder

Find open-access research across the web, download it in bulk, and bundle it into a tidy local library — ready to drop into any AI context window.

Website · Download · Developer guide


Document Finder started as a way to find and compress documents into context for AI and RAG, and grew into a broader tool for discovering and downloading open-access research. It searches several sources in parallel, downloads the papers, extracts their text, and stores everything in per-query SQLite libraries you can export as a single .zip. Native desktop app — Rust backend, Solid.js frontend, Tauri.

Features

  • Unified discovery across eight open-access scholarly sources plus built-in web search, with natural-language query expansion into sub-queries.
  • Live download stream — watch documents arrive in real time with throughput, ETA, file-type breakdown, and a per-source lane chart as the async Rust backend fetches, retries, and extracts text.
  • Plain-language results — every skipped or failed download explains itself in one sentence ("This source blocked the download — it may need a sign-in"), not an HTTP code.
  • AI-ready exports — bundle any library (PDFs, EPUBs, extracted text) into a .zip for a context window.
  • Built-in web meta-search — DuckDuckGo, Bing, Brave, Mojeek, Marginalia, and Startpage behind a per-engine circuit breaker, with an in-process SearXNG-compatible server and a public SearXNG pool as fallbacks. No Docker, no setup, no API keys.
  • Optional on-device AIbge-small reranks results and an on-device Qwen 2.5 1.5B expands queries and filters borderline hits; both downloaded on first use, run fully offline.
  • Editorial workstation UI — three themes (Paper / Slate / Midnight), nine accent colors, compact/regular density, and a stacked-or-split download stream, with reduced-motion support. Defaults to Slate + Sky blue.

Sources

All open-access. No API keys.

Source What it covers
arXiv Preprints in CS, physics, math, and more
OpenAlex ~250M scholarly works, open-access filtered
Semantic Scholar ~200M papers with PDF links
Europe PMC Life-sciences & biomedical literature
Internet Archive Books, papers, and scanned media
DOAJ Directory of Open Access Journals
Zenodo Open-access papers, datasets & software (CERN)
Project Gutenberg 70,000+ public-domain ebooks
Web Built-in meta-search (DuckDuckGo, Bing, Brave, Mojeek, Marginalia, Startpage) + SearXNG fallback

Installing a release

The release builds are not notarized (the macOS build is ad-hoc signed), so each OS asks you to allow the app on first launch:

  • macOS (Apple Silicon) — right-click the app → Open, then confirm. If macOS still says the app "is damaged and can't be opened", clear the download quarantine flag and reopen:
    xattr -dr com.apple.quarantine "/Applications/Document Finder.app"
  • Windows — SmartScreen → More infoRun anyway.
  • Linux — the .deb/.rpm/.AppImage need glibc 2.39+ (built on Ubuntu 24.04), so on older distros (Ubuntu 22.04, Debian 12, RHEL 9, Mint 21) use the Flatpak, which ships its own runtime and runs anywhere Flatpak does. If the window opens blank (some older GPU drivers + system WebKitGTK), launch with WEBKIT_DISABLE_DMABUF_RENDERER=1 set (add WEBKIT_DISABLE_COMPOSITING_MODE=1 if it persists) — or enable Settings → Safe rendering mode and relaunch, which sets the same workaround persistently.

Uninstalling

Document Finder is easy to remove cleanly on any OS.

  1. Erase its data (recommended, any OS). In the app: Settings → Danger zone → Erase app data. This deletes the downloaded AI models, caches, and run logs; tick the extra box to also delete your downloaded document library. This is the only step that knows a custom library folder, so do it first if you moved your library. Then quit the app.
  2. Remove the app itself:
    • Windows — Settings → Apps → Document Finder → Uninstall. The uninstaller offers a "Delete application data" checkbox (clears the %APPDATA%/%LOCALAPPDATA% caches incl. WebView2 storage); it never touches your Documents\Document Finder library.
    • macOS — quit, then drag Document Finder.app to the Trash.
    • Linuxsudo apt purge document-finder (.deb), sudo dnf remove document-finder (.rpm), delete the .AppImage, or flatpak uninstall --delete-data com.webworldwide.DocumentFinder.

Prefer the command line, or already deleted the app? Run the bundled script to clear all per-user data (it prompts before touching your library):

scripts/uninstall.sh          # macOS / Linux
scripts/uninstall.ps1         # Windows (PowerShell)
Exact data locations removed
What Windows macOS Linux
AI models + caches + config %APPDATA%\com.webworldwide.documentfinder ~/Library/Application Support/com.webworldwide.documentfinder ~/.local/share/com.webworldwide.documentfinder
Webview storage / caches %LOCALAPPDATA%\com.webworldwide.documentfinder ~/Library/{WebKit,Caches,Preferences}/com.webworldwide.documentfinder* ~/.config + ~/.cache/com.webworldwide.documentfinder
Run log %LOCALAPPDATA%\Document Finder\Logs ~/Library/Logs/Document Finder ~/.local/state/document-finder
Document library (kept by default) Documents\Document Finder ~/Documents/Document Finder ~/Documents/Document Finder

Build and run

Prerequisites: Rust, Node.js 22+, and a C++ toolchain with cmake + clang/LLVM (for the bundled llama.cpp). On Windows, install LLVM (or use Visual Studio Build Tools with the C++ and Clang components) so libclang is on your PATH.

./run.ps1         # Windows  (./run.sh on macOS/Linux — installs pnpm if needed)
pnpm tauri build  # native installers in src-tauri/target/release/bundle/

The first build compiles llama.cpp + ONNX Runtime from source (10–25 min). For a fast compile-check without the AI features (note: pnpm tauri build does not forward --no-default-features to cargo, so call cargo directly):

cargo build --manifest-path src-tauri/Cargo.toml --no-default-features --features=custom-protocol

To rebuild cleanly later — one command on any OS (Windows, macOS, Linux):

pnpm clean-build         # clean caches + release-fast build
pnpm clean-build:dev     # clean + hot-reload dev build

Data

Each search gets a folder under ~/Documents/Document Finder/library/:

your-query-slug/
├── library.db                ← SQLite metadata
├── _text/                    ← extracted plain text
└── paper-title-abc123.pdf    ← the downloaded files

library.db holds full metadata and is queryable with any SQLite client.

Contributing and license

See CLAUDE.md for the architecture map, build internals, release process, third-party license notices, and icon regeneration. New sources live in src-tauri/src/sources/ as modules implementing the Source trait.

Licensed under the GNU Affero General Public License v3.0. Copyright © 2026 Web World Wide.

About

An app to pull as many PDF resources as possible. Compresses into zip for upload into AI for specialized context

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages