Field note · 5 min read

    Bring your own model, without trusting it

    Teams want their own model for cost, contracts or data residency. What stays enforced whichever model answers, and the one thing we disclose instead of solving.

    5 min read4 sectionsWritten from shipped code

    01Why teams ask

    A team often already has a model contract: Azure OpenAI in their own tenant, Claude through Bedrock, a self-hosted model behind their firewall. They want enrichment to use it, so the spend lands on their bill and the text goes to a provider they have already approved.

    A workspace can name one: any OpenAI-compatible endpoint, the Claude API, Azure OpenAI, Vertex AI or Amazon Bedrock. Its calls are paid to that provider and aren't counted against the workspace's monthly AI limit.

    02What a model can't change

    The safety of the data was never the model's job, which is what makes swapping it safe. Whichever model answers:

    • It chooses only from the contract's allowed values; anything else is dropped, not stored.
    • Nothing below the field's confidence threshold is stored; it goes to a person.
    • Exclusions run in the query the agent's tool makes, so unknown is still never served as safe.
    • Every receipt names the model that answered, as the model and the host it was reached at.

    That is the same design as rules before AI: the model reads, the code decides.

    03Keys and addresses

    The credential is encrypted with AES-256-GCM, never returned by any endpoint, and only ever sent to the address it was saved for. An address on a private or internal network is refused, so a model setting can't be pointed at our own infrastructure.

    Search embeddings can come from the workspace's model too, with an OpenAI-compatible or Azure provider and an embedding model from a fixed list, checked on save. A release keeps the embedding model it was built with, and queries against it use the same one.

    04The part we disclose instead of solving

    Confidence thresholds were tuned against our own model. Another model's 0.8 may not mean what ours does, and on a safety field that matters. We haven't solved that, so the form says it plainly before anyone can save a model.

    Test the tool, not just the data

    An answer key says the values are right. It doesn't say the search an agent sends still answers right. Scenarios check that, and hold back a release that would break one.

    Read next