Removing the LLM From the Hot Path: Building SThree’s Skills Taxonomy Pipeline

Large language models have become the default answer to problems that do not always need them. I think that is bad engineering. At SThree, I found that search, fixed rules and a small classifier could reach about 96% measured precision without making a live model call for every skill.

The Skills Taxonomy pipeline reads CVs and attaches standardised skill labels that support recruiter search, candidate-role matching and talent pools. Errors have practical consequences. A CV that says "no Kubernetes experience" should not emit Kubernetes as a skill. SOAP should not be interpreted as a cleaning skill when a developer is describing a web-services protocol.

The system works across roughly 174,000 labels drawn from ESCO, O*NET, ACM CCS, the UK SSC catalogue and Canada's NOC. I designed, built and evaluated this end to end for SThree, working with its Skills Taxonomy team.

The system I replaced retrieved possible skills and asked a language model to approve or reject each one. It reached about 98% precision, but cost roughly £1,800 per million CVs and took 4 to 8 seconds per CV. I ran 30 experiments to see how closely a cheaper system could approach that result.

What replaced the language model

The first useful change was only three lines of Python. I sorted each CV's proposed skills by their historical error risk and kept the lowest-risk 20. That lifted precision by 12.03 percentage points over the 75% baseline and remained the largest single improvement in the programme.

I then combined that rule with six other inexpensive checks. They calibrated results by source, reduced the influence of unreliable risk estimates, removed weak skills that appeared only once, rejected risky matches in very short sentences, blocked a small set of known bad labels and handled document headers separately. Together they reached about 91% precision with no live model calls, a 16.44 point improvement over the baseline.

Once the fixed checks were in place, I measured what the remaining language-model calls actually changed. A four-line change reduced the rescue traffic from roughly 19 calls per CV to about 0.11, a 174-fold reduction, while measured precision changed by 0.00 points. The cheaper checks had already removed almost every case those calls would have rejected.

One result also changed how I thought about the evaluation. The first run of the V3-only pipeline on an external benchmark of 500 CVs from three sources reported 67.34% precision. It also reported a 22% hard-hallucination rate, meaning that the system emitted skills the CV clearly did not claim. It looked as though the system had failed to generalise. The problem was in document construction. Two of the three sources stored CV text without useful sentence boundaries, so the builder collapsed a CV into one long paragraph and the matcher began finding common-noun labels inside it.

A roughly 30-line sentence-splitting fallback raised precision on the same 500 CVs from 67.34% to 90.64%. Nothing about the model had changed. The apparent model failure was a text-processing bug.

The fixed checks took the system a long way, but they all worked with similar information. The final improvement came from a logistic-regression classifier that compares the meaning of a CV sentence with the proposed skill label. It uses embeddings from a small public sentence transformer, trains in under a second on a CPU and scores an emitted skill in about 0.2 milliseconds. The stored classifier is 7 kB.

I evaluated it by keeping every sentence from a given CV on the same side of the train-test split. At the main threshold, it added 4.2 precision points. A simpler split produced a larger 5.71 point result, but sentences from the same CV could appear in both training and testing, so I did not use that as the headline figure. About 31% of the training examples still share source-corpus conventions with the main evaluation set. The 4.2 point lift should therefore be treated as an upper bound on performance across completely different corpora.

What went into production

The live production path now uses retrieval, fixed filters, calibration and the 7 kB classifier. It makes no language-model calls while tagging a CV. On the final benchmark of 500 CVs across 43 categories, it reached about 96% precision with a hard-hallucination rate as low as 0.05%. A threshold controls whether the system favours returning more skills or a smaller, cleaner set.

Operating cost fell from roughly £1,800 to about £10 per million CVs, and processing time fell from 4 to 8 seconds to about 50 milliseconds per CV. Against the as-shipped 2024 system, that is about 180 times cheaper and 100 times faster. A modern version of the old system using prompt caching would narrow the estimated cost difference to roughly 30 to 60 times.

Expand

The new system does not beat the old one on every measure. The earlier version reached roughly 98% precision, about two points higher, and the two systems were measured on different benchmarks rather than in a direct comparison. Training costs are also excluded from the £10 steady-state figure. Those benchmarks indicate a large reduction in operating cost and latency alongside an estimated two-point precision difference, but they do not provide a direct comparison.

Other approaches added complexity without improving the result. Grey-zone language-model calls and a cross-encoder had no useful cases left after the filters ran, while stronger statistical priors changed almost nothing. Generated label descriptions improved recall but reduced precision, and a local 7B validator performed poorly on agreement and throughput.

The evaluation still has limits. One language model judged the emitted skills and supplied the labels used to train the classifier. The old 98% system was not rerun on the final 500-CV benchmark, so the two-point difference is cross-benchmark. The classifier score ranks candidates but is not a calibrated probability. The work also covers English-language CVs and Western skill taxonomies rather than multilingual recruitment.

Language models still had a useful offline role in labelling and evaluation. What the experiments did not justify was a model call for every proposed skill. I would use the same test again: measure what each call changes, then remove it if simpler software produces nearly the same result.