Deciding what a question is about

Before KIRHA can plan a single provider call, it has to hand the planner the right twenty tools out of more than eight hundred. The layer that does this treats the catalogue the way a compiler treats hardware: questions are the source language, provider tools are the machine code, and in between sits an intermediate representation of canonical actions. This is the story of that layer, and of how the frontier model at its centre was replaced by one we trained ourselves.

The planner needs tools in its prompt to plan with, and the obvious first version is to give it all of them. That version dies quickly. Put the whole catalogue in front of a model and the plan gets worse as the catalogue gets better: a longer prefill costs latency before the first token, and a model weighing hundreds of tool descriptions reasons longer and less well, because most of what it is reading has nothing to do with the question.

Our first fix was verticals, curated bundles of providers around a domain: crypto, company data, insurance. Route the question to a vertical, hand the planner only that vertical’s tools, and the context is bounded again. It worked, until it didn’t, and the way it failed was instructive: verticals are a unit of curation, not a unit of relevance. A good vertical keeps growing, and crypto alone accumulated enough providers that its tool list recreated the original problem one level down. Any fixed partition of a growing catalogue eventually has partitions with the same disease as the whole.

The realisation that unblocked us was that the planner never needed the tools’ definitions to be chosen, it needed the question’s intent to be understood. Those are different problems. Tool definitions are big because they carry schemas, parameters and provider quirks: everything execution needs and selection doesn’t. What selection needs is a compact vocabulary of the things the platform can do, independent of who provides them. So we built one.

#An intermediate representation for tools

A compiler does not translate C directly into instructions for every processor it supports. It translates into an intermediate representation, a small stable vocabulary of operations, and then lowers that IR to each backend. The economics are the point: front-ends and back-ends multiply independently, and the IR keeps the product of the two from becoming the cost.

Our catalogue has the same shape. On one side, unbounded phrasings in several languages: “who holds the most USDC on Base”, “top USDC holders base chain”, “plus gros détenteurs d’USDC sur Base”. On the other, many providers exposing overlapping capabilities behind different schemas, where three MCPs can each list token holders with three different parameter conventions. The IR between them is the canonical action: a verb-first name, get token holders, a one-line description, a handful of natural-language synonyms, the domains it lives in, and an embedding. Each action is linked in the graph to every provider tool that implements it, however many providers that turns out to be.

Translation, then, is compilation. A query comes in with its vertical; the system loads that vertical’s actions, actions rather than tools, and puts them in front of a model as one line each, name and description. The model picks the one to three action names that cover the intent, and everything after that is resolution rather than reasoning: an exact-match lookup from name to action, a graph hop from action to the tools that implement it, and a ranked list back to the planner. The tools’ schemas, the part that made the original context explode, never enter the selection prompt at all. That is the whole compression: the planner’s shortlist is chosen in a vocabulary two orders of magnitude smaller than the thing being chosen from.

The IR also gives disambiguation a place to live. Crypto tools cluster into ecosystem families, bitcoin, ethereum, multichain, and the same action can be implemented in several of them. Rather than splitting actions per family and inflating the vocabulary, the model appends a hint when the query names its ecosystem: get token holders, ethereum. The hint is not part of the action; it filters which backends the action resolves to. In compiler terms, the IR stays target-independent and the hint selects the backend.

Two more mechanisms round out a translation, both deliberately subordinate. If the model’s picks resolve to fewer tools than the caller’s budget, a vector search over action embeddings tops up the list with the picked actions’ nearest neighbours, and its scores are capped just below the model’s, so a similarity guess can never outrank a reasoned pick. And tools connected to the picks by learned relations, this one composes with that one, this one is an alternative to that, ride along in a separate slot, so the planner sees what the picks are likely to need next without the extras displacing anything.

There is a second consumer of the same IR. When no vertical is known yet, a discovery pass runs the same translation against the entire action vocabulary and asks the opposite question, not which tools but which vertical this question should be routed to, scored by how much of the query’s intent each vertical’s actions cover. The IR is what makes that affordable: the whole catalogue at one line per action fits where the whole catalogue at one schema per tool never could.

#A grammar that writes itself

None of this works if someone has to hand-author the vocabulary. Providers join weekly, so the mapping from their tools to canonical actions has to keep itself current. We call the whole living structure the grammar, and it maintains itself in three moves whenever an MCP syncs its tools.

• Extraction. Each new tool's JSON schema is flattened and shown to a model that answers four small questions: what verb, what entity, what domain, how confident. Parameter names carry most of the signal, since a coinId input says cryptocurrency and lookup-by-key more reliably than the prose description does.
• Matching or clustering. The extracted proto-action is compared against the existing vocabulary with a similarity that is deliberately more than embedding cosine: it blends the embedding with domain overlap and verb agreement, because two tools can sound alike in vector space while doing categorically different things. Score high enough against an existing action and the new tool simply attaches to it, so the grammar grows by reference and the vocabulary does not grow at all. The leftovers, tools genuinely new in kind, are clustered together.
• Naming. Each new cluster goes to a model once more and comes back as a canonical action: name, twenty-word description, and three to seven synonyms phrased the way a person would ask. The action's embedding is the centroid of its members, and it joins the graph.

The graph is the substrate under all of it: verticals include MCPs, MCPs own tools, actions represent tools, and tools carry relation edges to other tools, composes-with, prefers, alternative-to. The relation edges get the most paranoid treatment in the pipeline, because a wrong edge pollutes every future translation that touches it. Candidates are generated deterministically from schema compatibility, and a verifier model judges them under an explicit instruction to reject when in doubt, with the calibration spelled out: if you are accepting more than three quarters of a batch, you are being too lenient.

#Trading the frontier model for our own

The one model call on the hot path, the intent parse at the heart of translation, ran on a frontier API. The task is narrow: read a list, pick one to three names. Narrow tasks on hot paths are what fine-tuning is for.

The constraint that shaped the training is the same one that shaped the grammar, that the vocabulary is alive. A model that learns “this query means that action” is obsolete the first time an action is renamed. So the dataset is built to make memorisation a losing strategy. Every query appears with several different candidate lists. In half the examples the action names are deliberately shuffled onto the wrong entries, descriptions stay truthful, and the gold answer is whatever name the right description happens to be wearing, so reading is the only strategy that survives. Fabricated actions that never existed appear as correct answers, teaching the act of selecting something never seen before. And eighteen real actions were held out of training entirely; queries answered by those eighteen form the evaluation we trust most, because they measure the model doing the thing the catalogue will demand of it every week: picking an action it has never encountered, from its description alone.

The first fine-tune scored an F1 of 0.75 against held-out production labels, and then three successive dataset improvements moved it nowhere. The step that explained the wall cost almost nothing: we ran the teacher itself against our evaluation, same prompts, same metrics. It scored 0.443. The labels are single samples from a model that agrees with its own majority answer less than half the time, which means 0.75 was not a plateau under the ceiling, it was the ceiling, set by label noise. The student had been outscoring its teacher by thirty points while we tried to fix it. Before judging a distilled model against its labels, measure what the teacher scores against them.

With the pipeline proven we went hunting for the smallest model that holds the line, training each candidate on the identical dataset and seed.

Every student clears the teacher by roughly thirty points in distribution and matches it on never-seen actions, which reads as the intrinsic difficulty of picking an unfamiliar action from one description, since scale does not move it in either direction. The result that matters is the last row: a 0.8B model at parity with the 4B. Its 1.3-point edge sits inside the noise of a 196-query evaluation, so parity is the claim we stand behind, with the best French score of anything we trained, at a size that serves on the smallest GPU a cloud rents, or on a laptop.

All four numbers are our own evaluations against production labels rather than a public benchmark, so they show the shape of the result rather than ranking these models in general.

Two properties of the deployment are worth more than the scores. The model’s output is grammar-constrained: decoding is restricted, per request, to exactly the candidate action names, so an out-of-vocabulary answer is impossible by construction rather than unlikely by training. And because the model grounds on descriptions rather than memorised names, the catalogue text is configuration. Rewrite an action’s description and the model’s behaviour changes on the next request, with no retraining. We have already fixed production confusions both ways: once by editing a description, once by removing the empty answer from the grammar itself.