A command-line search engine built in C++17 from scratch — no external libraries, no frameworks. It scans a folder of plain text documents, builds an inverted index, and returns TF-IDF ranked results with contextual snippets.
This project was built to demonstrate hands-on understanding of data structures, ranking algorithms, file I/O, text processing, and clean modular C++ architecture.
- Inverted index — maps every unique term to the documents containing it for fast lookup without scanning every file on every query
- TF-IDF ranking — scores results by how often a term appears in a document relative to how common it is across the whole corpus, so the most relevant documents rise to the top
- Smoothed IDF formula — prevents zero scores when a term appears in every document (
log((1+N)/(1+df))+1) - Text processing pipeline — lowercasing, punctuation removal, tokenization, and stop word filtering before indexing
- Contextual snippets — shows the surrounding passage where your query term appears, with matched terms wrapped in
[brackets] - Index statistics — vocabulary size, total token count, average document length, top 10 most frequent terms
- Auto-indexes on launch — documents are loaded and ready the moment the program starts, no setup step required
- Graceful edge case handling — empty queries, stop-word-only queries, no matching documents, and empty files are all handled cleanly with informative messages
| Language | C++17 |
| Build System | CMake 3.14+ |
| Key STL containers | unordered_map, vector, string, filesystem |
| Testing | GoogleTest (optional — off by default, no install required to build) |
| Dependencies | Standard library only |
Prerequisites: CMake 3.14+ and a C++17 compiler (GCC 8+, Clang 7+, MSVC 2019+).
git clone https://github.com/juliavolpe/mini-search-engine
cd mini-search-engine
mkdir build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Release
makeFrom the project root:
./build/mini-search-engineThe program auto-indexes the documents/ folder and drops you straight into the menu.
=============================
Mini Search Engine
=============================
Indexes local .txt files and ranks results by TF-IDF relevance.
Loading documents from 'documents'...
Ready! Indexed 8 documents.
Example searches: hockey | machine learning | electric vehicles | investing
-----------------------------
What would you like to do?
1 - Search for a keyword or phrase
2 - Show index stats
3 - Re-index documents
4 - Exit
Enter a number (1-4): 1
Search query: machine learning algorithms
[1] technology.txt (score: 0.0591)
Terms: machine, learning, algorithms
"...enabling [machine]s to perform tasks that once required human intelligence.
[Machine] [learning] [algorithms] analyze massive datasets to identify patterns..."
[2] cars.txt (score: 0.0358)
Terms: machine, learning, algorithms
"...with sensors, cameras, and [machine] [learning] [algorithms] enabling
vehicles to navigate roads with minimal human input..."
Enter a number (1-4): 2
--- Index Stats ---
Documents indexed: 8
Unique terms: 937
Total tokens: 1501
Avg document length: 187.6 tokens
Top terms:
that (16)
like (12)
from (11)
players (6)
health (5)
The project is organized as a static library + executable. All search logic lives in libsearchengine so it can be built and tested independently of the CLI.
mini-search-engine/
├── include/ # Public interfaces — one header per class
│ ├── Document.h # Data struct: id, filename, content, tokens, totalTokenCount
│ ├── SearchResult.h # Result struct: filename, score, matchingTerms, snippet
│ ├── InvertedIndex.h # Posting struct + unordered_map<term, vector<Posting>>
│ ├── TextProcessor.h # Static pipeline: normalize → tokenize → stop word removal
│ ├── DocumentLoader.h # Reads .txt files from a directory using std::filesystem
│ ├── SnippetGenerator.h # Extracts context window around first query match
│ └── SearchEngine.h # Orchestrator: owns the index, scores queries, returns stats
├── src/
│ ├── main.cpp # CLI menu — all user interaction lives here
│ └── *.cpp # One implementation file per class
├── documents/ # Sample corpus (sports, tech, finance, travel, health, gaming, cars, cooking)
└── tests/ # GoogleTest unit tests (optional)
Data flow:
DocumentLoader::loadDocuments()
→ raw Document objects (content set, tokens empty)
SearchEngine::indexDocuments()
→ TextProcessor::process(content) returns filtered tokens
→ Document.totalTokenCount set from raw (pre-filter) token count
→ InvertedIndex::addDocument() builds the posting lists
User query
→ TextProcessor::process(query) returns query terms
→ InvertedIndex::getPostings(term) returns candidate documents
→ calculateTfIdf(term, doc) scores each candidate
→ SnippetGenerator::generateSnippet() adds context window
→ Results sorted by score descending
TF-IDF is the standard baseline algorithm for text relevance ranking. It rewards terms that appear frequently in a specific document but rarely across the whole corpus — so rare, meaningful words score higher than common ones.
Term Frequency (TF) — how often the term appears in this document, normalized by total word count:
tf = termFrequency / totalTokensInDocument
Inverse Document Frequency (IDF) — how rare the term is across all documents. Smoothed to prevent zero scores when a term appears in every file:
idf = log( (1 + totalDocuments) / (1 + documentsContainingTerm) ) + 1
Final score — summed across all query terms:
score(doc) = Σ tf(t, doc) × idf(t) for each query term t
A word like "hockey" that appears 3 times in a 100-word document and only in 1 of 8 files will score significantly higher than a word like "players" that shows up across multiple documents.
mini-search-engine/
CMakeLists.txt
README.md
documents/
sports.txt technology.txt finance.txt travel.txt
health.txt gaming.txt cars.txt cooking.txt
src/
main.cpp TextProcessor.cpp InvertedIndex.cpp
DocumentLoader.cpp SnippetGenerator.cpp SearchEngine.cpp
include/
Document.h SearchResult.h TextProcessor.h
InvertedIndex.h DocumentLoader.h SnippetGenerator.h
SearchEngine.h
tests/
TextProcessorTests.cpp SearchEngineTests.cpp
Tests are optional and off by default so the project builds cleanly without any external dependencies. To run them:
# macOS
brew install googletest
cd build
cmake .. -DBUILD_TESTS=ON
make
./run_testsTextProcessor tests — normalize, tokenize, stop word removal, empty input handling
SearchEngine tests — search before indexing, empty query, stop-word-only query, no matching terms, positive TF-IDF score for a known term
- Phrase search — match
"electric vehicles"as an exact ordered sequence - Boolean queries — support
hockey AND team NOT soccer - Stemming / lemmatization — normalize
"running","runs","ran"to the same index entry - Fuzzy matching — handle typos and spelling variants
- Persistent index — serialize the inverted index to disk so re-indexing isn't required on every launch
- Multithreaded indexing — process documents in parallel using
std::threadorstd::async - Web crawler input — accept URLs as input sources instead of local files
- Search result highlighting — terminal color codes to highlight matched terms inline
- GUI or web API — expose search over HTTP or build a desktop interface
This project demonstrates:
- C++17 —
std::filesystem, structured bindings,constcorrectness,static_castfor locale-safe character handling - Data structures — inverted index using
unordered_map<string, vector<Posting>>for O(1) term lookup - Algorithms — TF-IDF ranking, sorting, substring search, sliding context window
- File I/O — directory traversal, file reading, handling edge cases like empty and non-text files
- Text processing — normalization, tokenization, stop word filtering
- Modular architecture — static library + executable, single-responsibility classes, clean header/source separation
- Build tooling — CMake with library target, release config, optional test target
- Code quality — no raw pointers, no external dependencies, zero warnings, graceful error handling