Control tests

Sets the bar. IIA practitioner checklist

Method. Reconcile the declared inventory against the model call sites actually present in source, confirm every entry names an owner and a classification, and enumerate the providers and machine credentials the inventory should account for

Finding. The inventory now exists, names an owner and a classification for each attachment, and reconciles to the code. This test is the control rather than a one-off check: a fourth call site added without declaring it here fails CO1 instead of passing unnoticed. What it still does not do is reconcile against egress to model providers, which would catch a call made outside this module.

Sets the bar. AI Act Articles 6, 49

Method. Read the declared classification, confirm it reaches a conclusion with reasoning, an owner and a date, and re-perform the Annex III test on the lending use case against it

Finding. The classification is genuinely arguable, which is the reason it had to be written down rather than assumed. It concludes high risk under Annex III 5(b) and shows the step that decides it: the Article 6(3) derogation reads as available for a preparatory task, and the profiling subparagraph closes it. An assessor who disagrees now has a specific step to disagree with.

Sets the bar. AI Act Article 25

Method. Test whether fine-tuning, retrieval augmentation or prompt layering has occurred, then read the role assessment and confirm it carries a date and addresses each modification the code contains

Finding. Two roles run at once and the assessment separates them: provider of the Cadence system, deployer of the 2 models inside it. Prompt layering is named and dismissed as the deciding factor, and Article 25(1)(c) is what carries the conclusion. The consequence an institution would care about is recorded: Article 26 duties cannot be handed back to Groq or Google, so a conformity claim from either discharges nothing.

Sets the bar. AI Act Article 27

Method. Confirm the classification requires one, then reconcile the assessment against the 6 matters Article 27(1) lists and confirm each is answered

Finding. The assessment answers all 6 matters and declares 3 gaps in itself rather than reading as complete. The one worth carrying forward is under (d): categorisation accuracy is measured overall and never by applicant group, so the risk that error falls on irregular income and unusual salary vocabulary is reasoned rather than quantified. Article 27 asks about harm to specific groups, and an overall accuracy figure cannot answer that question.

Sets the bar. BaFin lifecycle guidance; ECB drift expectation

Method. For every attachment in the inventory, require a defined threshold, a named owner of it, and a stated consequence on breach, then record how each consequence is applied

Finding. Partial, and the missing half is enforcement. Every attachment carries a level, an owner and a consequence, which is the part of the objective a register can satisfy by being written. Automatic enforcement stands at 0 of 3. A push to main deploys whatever the branch holds, so every consequence in this register reaches the system through a person who runs a check and acts on the result. scripts/verify-rationale.ts computes a pass or fail without an API key and comes closest to a gate, and a person runs it from the command line. The judge threshold is the weakest, applied by review with no script behind it, and that one is deliberate: the judge has never been measured against human labels, so a person keeps an unvalidated scorer out of the decision. An action a person has to remember is weaker than the objective describes, which is why this reads partial rather than pass. The categoriser threshold also fails at the job it is there for. Scored group by group, the model runs 12.7% and 32.8% below the rest of the population for the 2 groups a wrong label hurts most, while the overall figure reads 96.3% and the gate stays quiet. A threshold set on an average cannot react to a disparity underneath it. Recorded as F8.

Sets the bar. AI Act Article 26; GDPR Article 22 after C-634/21

Method. Confirm an override path exists, that the reviewer receives reasoning rather than only a recommendation, and measure the override rate

Finding. The capability is real and now measurable, which it was not before this work. It is not yet evidenced: there is no recorded population of decisions to compute a rate over, and the quality of the reasoning the reviewer receives has never been validated against human judgement.

Sets the bar. BaFin lifecycle guidance

Method. Re-performance: derive a cache key for the same transaction under 2 model ids and assert they differ, then under 2 prompt versions and assert the same, so either change is a miss rather than a silent reuse. Then confirm the prompts carry an explicit version marker.

Finding. Both change paths now work and the test re-performs both. A label is stored against the model and the prompt that produced it, so a corrected exemplar reaches lines already labelled instead of stopping at new ones. What remains outside the control is the provider: a model updated silently behind a stable model id changes the answer without changing the key, and nothing here would detect it.

Sets the bar. BaFin access control expectation; OWASP LLM06

Method. Identify non-human identities holding provider credentials and test for vaulting, time-bounding, session logging and recertification

See the evidence. Nothing to link. Provider credentials are server-side and are not exposed by the running system.

Finding. There is no agent here, so the excessive-agency risk does not arise. The machine identities that do exist hold unexpiring API keys in environment variables, which would not meet a privileged-access standard in production.

Sets the bar. AI Act Article 86

Method. Re-performance: produce a declined decision and read back the explanation the applicant would receive, then assess whether it is meaningful rather than merely present

Finding. This is the strongest control in the system. An explanation exists, is grounded in the decision package, states the outcome and the reason, and the console drills every figure to its source transactions. What has not been tested is whether a person who is not a loan officer would find it meaningful, which is the standard Article 86 actually sets.

Sets the bar. AI Act Article 50

Method. Inspect the applicant-facing journey for disclosure of automated processing and of AI involvement specifically

Finding. Two disclosures answering to 2 different instruments are now both made on the decision screen. GDPR Article 22 is served by telling the applicant a decision was automated and offering review. Article 50 is served by naming what the model did and what it did not do, which matters here because the model touches the inputs and the explanation while the outcome is arithmetic. Neither disclosure has been tested for comprehension by anyone outside this build.