AI-Assistant for Sound Design
This project implements a local RAG-esque pipeline to provide fast, accurate search results without the need for a backend server.
The python pre-processor uses
pymupdf to extract the text from a .pdf file. The text content is split into sentence-based chunks and sanitized. The chunks are processed by sentence-transformers to then fully convert the .pdf -> vector embeddings (.json).
The frontend synth-search uses Transformers.js to perform local inference. When a user searches, the query is vectorized in-browser using the AI model all-mpnet-base-v2. A cosine similarity calculation is performed against the local index to find the most contextually relevant matches.
The semantic data matched from the user query is sanitized and processed as a prompt to the AI model Flan-T5 Base. The text-to-text generation is cleaned and sent to the user.
- Synth Secrets by Sound On Sound
- Welsh's Synthesizer Cookbook by Fred Welsh
This project is released under the GNU GPL License - see the LICENSE file for details