Generative AI features inside your software, tested and costed before launch

Generative AI can draft, summarise, rewrite, translate, classify and extract information inside the screens your users already work in. Typical examples are a summarise button on a long case record, first-draft replies in a helpdesk, product copy generated from attribute data, or incoming documents turned into form fields. We expose each capability through an internal API, so it can be reused across web, mobile and back-office tools and measured on its own.

Most projects should start with prompt engineering and fine-tune only when evaluation shows a clear gap. Well-designed prompts with good examples and structured output often meet the quality bar at lower cost and with more flexibility. Fine-tuning becomes worthwhile when you need a very consistent format or style, high volumes on a smaller and cheaper model, or behaviour that prompts cannot hold reliably. We decide by scoring both approaches on the same test set.

Hallucinations cannot be removed entirely, so we limit what the model can invent and check its output against the source data before anyone relies on it. The model receives only the data it should use. Generated text is constrained to fields supplied in the input, claims such as prices, dates or specifications are validated by code, and missing information is flagged rather than filled in. For customer-facing content, a review step lets a person approve or edit each item before it is published.

We define a schema for the output, use the provider's structured output features and validate every response in code before it is stored. If a response fails validation, the system retries with the error message or routes the item to a person. Extraction quality is measured field by field against labelled examples, so you can see which fields are dependable and which still need human checking.

We run every candidate on the same set of real examples from your business and compare the scores side by side. Each output is graded against your written definition of good output, alongside latency, the rate of malformed structured responses and cost per request at your expected volume. Open-weight models join the comparison when self-hosting is an option, with GPU hosting added to their cost. Where scores are close, a blind review by your team decides. When a provider releases a new version, we rerun the same comparison rather than starting again.

We set a cost budget per feature before launch and track spend against it from the first request. The largest savings usually come from sending less: trimming context to the fields the task needs, capping output length and ordering prompts so the provider's prompt caching applies to the repeated part. Non-urgent work such as overnight summaries can use discounted batch endpoints where providers offer them. The gateway attributes every call to a feature and customer, and alerts fire when cost per successful result drifts.

Not under the business API terms we build on: the OpenAI API, Azure OpenAI and Google's paid Gemini API do not use your prompts or outputs to train models by default. Consumer chat apps and some free tiers work differently, which is one reason staff should not paste customer data into them. Providers may still keep requests for a limited period for abuse monitoring, so we also mask personal fields, redact logs and confirm each provider's current retention terms during discovery.

An LLM gateway is a service between your application and the model providers that handles API keys, rate limits, retries, caching, logging and failover in one place. You need one once several features or teams call models, or when spend must be attributed per customer. For a single internal tool, a thin wrapper inside the application is enough to start. Because every feature calls one internal endpoint, changing provider becomes a configuration change followed by an evaluation run, not a rewrite.

We treat every piece of user-supplied or external text as untrusted and design the feature so an injection cannot cause harm even if it gets through. That means separating instructions from content, filtering suspicious inputs, limiting what model output is allowed to trigger and validating responses before they reach users or other systems. Injection attempts are part of our red-team test set and are rerun whenever prompts or models change.

Our LLM gateway can fail over to an alternative model or provider, so the feature degrades gracefully instead of stopping. Because prompts and evaluation sets are versioned separately from application code, moving to a new model means rerunning the test suite, adjusting prompts where scores drop and releasing through the normal pipeline. Model retirements become planned upgrades rather than emergencies.

Timelines depend mainly on how clearly good output can be defined, how much example data exists and how deeply the feature touches existing systems. A single, well-scoped feature such as summarisation or extraction inside an existing application is often in a staged release within a few weeks of discovery. Features that need fine-tuning, new data pipelines or several integrations are planned in phases, with working software reviewed every one to two weeks.

Cost depends on scope, so we give a fixed build quote after discovery rather than a list price. The model call itself is a small part of the build. Most effort goes into the work around it: assembling evaluation data from your records, structured output and validation, guardrails, the interface and the integrations. Running cost is model usage plus hosting, projected from your request volumes and the cost per request measured during model comparison, so each feature's monthly figure is known before launch.

You own the code, prompts, evaluation datasets and configuration we build for your project, and they are handed over with documentation. Prompts and test sets live in your repository like any other source file, so your team can review changes and rerun evaluations without us. Model API accounts can sit under your organisation from the start, which keeps billing and access in your control.