Learn

The categoriser & how it is evaluated

A language model reads the statement and labels what each transaction is. It produces structured categories and grounded rationale — it never produces the lending decision. Its quality is then measured against an independent labelled set.

Scope

What the model does, and what it does not

It categorises and explains

The model maps each raw transaction — a counterparty, an amount, a date, a terse reference string — to a category and a short rationale grounded in that line's own evidence. Recurrence and obligation flags turn noisy text into structured signal the affordability engine can sum.

It never decides

The model has no view on approve, refer or decline. Those follow from deterministic rules over the categorised totals. Confining the model to perception keeps the decision auditable and keeps model error contained to a single, measurable stage.

Taxonomy

Sixteen categories

Every transaction resolves to exactly one of these. The same fixed taxonomy is shared by the data generator's ground-truth labels and the model's output, which makes the two directly comparable.

SalaryRecurring incomeRentUtilitiesInsuranceGroceriesTransportSubscriptionsLoan repaymentBNPLOther creditDiscretionaryTransferFeesGamblingOther

Contract

A validated output schema

The model is constrained to a typed schema, validated on the way out. A response that does not parse is rejected rather than trusted, so malformed or hallucinated structure never reaches the engine.

CategorisationResult

categoryenum (16)One of the sixteen taxonomy labels.

subcategorystringFree-form refinement of the category.

confidencenumber 0–1The model's self-reported certainty.

isIncomebooleanWhether the line is incoming funds.

isRecurringbooleanWhether it repeats on a regular cadence.

isObligationbooleanWhether it is a contractual debt.

Evaluation

Measured against an independent ground truth

The categoriser is run over a labelled synthetic set and scored on accuracy, per-category precision and recall, and a confusion matrix.

Why the measurement is non-circular

The labelled set is produced by a generator that assigns each transaction its true category at creation time — before the model sees anything. Ground truth therefore comes from a source entirely independent of the categoriser. The model is graded against labels it had no hand in writing, so a high score reflects agreement with the generator rather than the model marking its own work.

Open the model eval

Honesty

The limitation, owned plainly

Synthetic labels measure consistency against a known generator, not absolute truth. A model that perfectly matches the generator has learned to agree with the generator's conventions — which is a far easier target than the genuine ambiguity of real statements, where an ambiguous transfer or an unlabelled standing order has no single objectively correct answer.

A production system would be evaluated on human-reviewed labels drawn from real data, with inter-rater agreement reported so the irreducible disagreement between expert annotators is visible alongside the model's score. This demo is explicit that it measures the former, not the latter.

How the AI is evaluated