Football Player Search Engine

A Python-based information retrieval system built to search and rank football player biographies. Given a natural language query, the engine returns the most relevant players from a dataset of 60 HTML biography pages using classic IR techniques.

---

Features

• HTML Parsing - Extracts structured player data (name, nationality, club, position, DOB) from 60 biography pages using BeautifulSoup
• Text Preprocessing - Lowercasing, stopword removal, and Porter stemming via NLTK
• Inverted Index - Efficient token-to-document mapping built from preprocessed documents
• TF-IDF Ranking - Scores and ranks documents by relevance to the user query
• Query Expansion - Synonym mapping (e.g. "defender" -> "centreback", "fullback") plus Named Entity Recognition (spaCy) to detect player names, clubs, and countries in queries
• Title Boosting - Boosts relevance scores when query terms match a player's title
• Precision@10 Evaluation - Measures accuracy of top 10 results against ground truth labels

---

Tech Stack

Tool	Purpose
Python 3	Core language
BeautifulSoup4 + lxml	HTML parsing
NLTK	Stopword removal, Porter stemming
spaCy (en_core_web_sm)	Named entity recognition
JSON	Data storage (documents, index)
CSV	Ground truth labels for evaluation

---

Project Structure

├── code/
│   ├── main.py            # HTML extraction and metadata parsing
│   ├── preprocess.py      # Tokenisation, stopword removal, stemming
│   ├── build_index.py     # Inverted index construction
│   └── search.py          # Query interface, TF-IDF ranking, evaluation
├── data/
│   ├── soccer_html/       # 60 footballer biography HTML pages
│   ├── documents.json     # Extracted player metadata
│   ├── preprocessed_documents.json
│   ├── inverted_index.json
│   ├── soccer.csv         # Ground truth labels for evaluation
│   └── search_results.txt # Persisted query results

---

How to Run

1. Install dependencies

pip install beautifulsoup4 lxml nltk spacy
python -m spacy download en_core_web_sm

2. Extract documents

python3 code/main.py

3. Preprocess and build index

python3 code/preprocess.py
python3 code/build_index.py

4. Run the search engine

python3 code/search.py

You'll be prompted to enter a search query (e.g. fast left sided defender) and an expected nationality label (e.g. english). The engine returns the top 10 ranked results and prints a Precision@10 score. Results are also saved to data/search_results.txt.

---

Notes

• HTML files are parsed with ISO-8859-1 encoding to handle special characters in international player names
• Ground truth CSV labels are used for evaluation only, not for ranking
• The pipeline is domain-independent and can be adapted to other HTML document collections with minimal changes

---

Author

Alexander McKenna