LLM & Agents — 2025

Multi-Agent Support SystemLangChain / LangGraph + RAG

Most support bots are one model doing everything badly. This is three specialised agents that hand work to each other, with a measured report on how often each one is right.

Role
Sole author — design, build, evaluation
Client
MSc Intelligent Systems · Distinction 92
Year
2025
Discipline
LLM & Agents

The problem

Human-handled support runs $15–25 per interaction, can't absorb launch spikes, and varies with whoever picked up the ticket. But the naive replacement — a single LLM answering everything — fails exactly where it matters: it can't tell a password reset from a billing dispute, and it escalates without context, so the human starts cold.

The approach

A LangGraph workflow over three specialised agents. The Inquiry Agent classifies into five categories with TF-IDF n-gram features and a Naive Bayes classifier, and runs sentiment analysis for priority. The Resolution Agent attempts an answer with retrieval-augmented generation over the knowledge base, and — critically — reports a confidence score rather than a flat answer. Below threshold, the Escalation Agent routes to the right human specialisation and assembles a full context package so nobody restarts the conversation. Every stage writes metrics to the database.

The outcome

92.5% overall categorisation accuracy (94% technical, 94% general, 93% complaints, 91% billing, 90% product). 65% of inquiries resolved automatically at 0.87 mean confidence, with an 88% resolution-success rate; the 35% escalated carried 0.62 mean confidence and were routed to the correct specialisation 95% of the time, with 100% context completeness. 2.8s per inquiry, 2.3 minutes to resolution, 50 inquiries/minute at peak. Graded 92 — Distinction.

Results

By the numbers.

Measured, not estimated
92.5%
Categorisation accuracy
65%
Resolved without a human
95%
Escalations routed correctly
Stack

What it's built on.

8 components
Python LangChain LangGraph RAG scikit-learn TF-IDF Naive Bayes PostgreSQL
Contact

Let's build
something honest.