A small internal-tool-style pipeline that reads a support ticket (subject + body), predicts its category — Billing / Technical / HR / General — and returns a confidence score plus a priority tag, exactly the way a real triage system in front of a live queue would need to.
tickets_dataset.csv— 102 hand-written dummy tickets, roughly balanced across the 4 categories.ticket_classifier.py— the full pipeline: cleaning → TF-IDF → training → evaluation → live prediction.requirements.txt— dependencies.
pip install -r requirements.txt
python ticket_classifier.py # trains, evaluates, and predicts on 7 unseen sample tickets
python ticket_classifier.py --demo # also opens an interactive CLI: type a ticket, get a live predictionText is lowercased, stripped of punctuation/numbers/URLs, and cleaned with a small
custom stopword list that deliberately keeps negation words ("not", "cannot")
since phrases like "not working" carry the actual category signal. Cleaned text
is vectorized with TF-IDF (unigrams + bigrams) — chosen over raw
Bag-of-Words because it downweights words common to every ticket ("please",
"account") and upweights words distinctive to a category ("invoice", "crash",
"leave"). Both Multinomial Naive Bayes and Logistic Regression are
trained and compared on a held-out test split (accuracy, precision/recall,
confusion matrix); Logistic Regression is used in production here because it
gives usable predict_proba confidence scores, not just a label.
Edge cases: if a ticket's cleaned text shares no vocabulary at all with
training data, the model has zero real signal, so it's defaulted to General
and flagged for review rather than letting the classifier guess. Any prediction
with confidence below 60% is similarly routed to a "needs human review" flag
instead of being auto-assigned — this is what happened for an off-topic
"do you have a Chennai office" ticket in testing (43.9% confidence), while a
clear billing complaint scored 72.8%. A lightweight keyword layer ("urgent",
"down", "not working", "crash") separately tags a ticket Urgent/Normal,
independent of category, so an urgent Technical ticket doesn't get buried.
With a larger, real ticket dataset I'd move past hand-picked stopwords to a
proper NLP pipeline (lemmatization, a standard stopword corpus) and add
cross-validation instead of a single train/test split, since 102 examples makes
any one split somewhat noisy — especially for General, the category with the
least distinct vocabulary and the most confusion with HR. I'd also calibrate
the confidence scores properly (e.g. CalibratedClassifierCV) rather than
relying on raw logistic regression probabilities, and add a feedback loop where
human-reviewed tickets get folded back into training data so the model improves
over time instead of staying static.