AI Products
AI features with accuracy you can measure before launch
We build retrieval, extraction, classification, and assistant features against a labelled evaluation set, so you know the error rate before the feature reaches a customer and you can see whether each change makes it better or worse.
- Typical timeline
- 6 to 12 weeks
- Starting point
- Evaluation set and baseline
- Deployment
- Your cloud or ours
(01)Overview
What this covers
The difficulty with language model features is not building a first version. A usable prototype takes days. The difficulty is knowing whether it is right often enough to put in front of a customer, and knowing that it stays right when the model, the prompt, or the underlying data changes.
We treat that as a measurement problem. Before building, we assemble a labelled set of real cases drawn from your own data and define what a correct answer looks like for each one. That set becomes the baseline. Every subsequent change is scored against it, which turns prompt and retrieval decisions into evidence rather than opinion.
The second half of the work is containment. Features are built so that low-confidence cases are flagged rather than guessed, sources are cited so a reviewer can verify an answer quickly, and a human review path exists for the categories where an error carries real cost. We state plainly what the system does not handle well, and that goes into your documentation.
(02)Capabilities
Capabilities
Scoped to the decision or task the feature supports, not to a model vendor.
- 01
Retrieval over your documents
Search and question answering across contracts, manuals, policies, tickets, and knowledge bases. Chunking, indexing, and reranking are tuned against your corpus, and every answer carries citations back to the source passage.
- 02
Structured extraction
Converting unstructured documents into validated records. Output is constrained to a schema, checked against your reference data, and anything that fails validation is routed to review rather than written downstream.
- 03
Classification and routing
Categorising tickets, documents, leads, or transactions against your taxonomy, with a confidence threshold below which the item goes to a person. Thresholds are set from measured data, not guessed.
- 04
Task-scoped assistants
Interfaces that answer questions or carry out defined actions within a bounded scope, with permissions enforced in the application layer rather than in the prompt, and with every action logged.
- 05
Evaluation harness
A labelled test set, scoring code, and a report that runs on every change. This is delivered to you as part of the codebase so accuracy can be verified independently after we finish.
- 06
Cost and latency control
Model selection per task, caching, batching, and fallback to smaller models where accuracy allows. Per-request cost and response time are tracked alongside accuracy so the three can be traded off deliberately.
(03)What you get
What you receive
Source code, evaluation data, and measured results, all under your ownership.
- The feature deployed in an environment you control, with infrastructure defined as code
- A labelled evaluation set built from your data, with the scoring criteria documented
- A baseline accuracy report covering overall performance and the categories where it is weakest
- Prompt and retrieval configuration under version control, with the result of each change recorded
- A review interface for the cases held back by confidence thresholds
- A written statement of known limitations, suitable for internal and customer-facing documentation
(04)Process
How we work
Measurement comes before building. If the evaluation shows the approach will not reach a usable accuracy level, you find out in week two rather than month four.
- 01
1.Scope and evaluation set
We define the specific decision the feature supports and what correct output means. Together with your team we label a representative set of real cases, including the awkward ones. This is the acceptance criteria for everything that follows.
- 02
2.Baseline
We build the simplest implementation that could work and score it against the set. The result tells us whether the problem is tractable, where the errors concentrate, and whether the remaining gap is worth the engineering cost.
- 03
3.Build to threshold
Retrieval, prompting, validation, and fallback behaviour are improved in measured increments until the feature clears the agreed threshold on the evaluation set. Each change is recorded with its effect on accuracy, cost, and latency.
- 04
4.Deploy and observe
The feature ships behind a flag to a limited audience first. We monitor real usage against the evaluation baseline, adjust thresholds from live data, and widen the rollout once the measured behaviour holds.
(05)Technology
Technology
Model and vendor choices are made per task and documented, so a change of provider is a configuration decision rather than a rewrite.
- Models
- OpenAI
- Anthropic
- Google Vertex AI
- Self-hosted open weights
- Retrieval
- pgvector
- Qdrant
- Elasticsearch
- Hybrid search and reranking
- Application
- Python
- FastAPI
- TypeScript
- Next.js
- Operations
- Langfuse
- OpenTelemetry
- Docker
- Terraform
(06)Questions
Questions we are asked
Will our data be used to train a model?
Not unless you ask for it. We use providers under enterprise terms that exclude training on submitted data, and we can deploy open-weight models inside your own infrastructure where contractual or regulatory requirements make that necessary. Data handling is documented before any production data is processed.
What accuracy can we expect?
That depends on the task and the quality of your data, and anyone quoting a number before seeing either is guessing. The baseline stage exists to answer this question with evidence from your own cases, usually within the first two to three weeks, and it is a defined point at which you can stop.
What happens when the model gets an answer wrong?
Features are designed on the assumption that some answers will be wrong. Low-confidence cases are held for review rather than acted on, answers carry citations so a reviewer can verify them quickly, and actions with real consequences require confirmation. The evaluation report tells you which categories are least reliable.
Can this run without sending data to a third party?
Yes. Open-weight models can be deployed in your own cloud account or on-premise. Accuracy for a given task may differ from a hosted frontier model, and we measure that difference on your evaluation set so the trade-off is a decision rather than an assumption.
Who owns the evaluation set and the code?
You do. The evaluation set, scoring code, prompts, and application code are delivered in your repository. You can verify accuracy, change providers, or hand the work to another team without depending on us.
Other services
Start a project
Begin with a baseline
Bring a task and a sample of real cases. We will tell you what is measurable, what accuracy looks like today, and whether the feature is worth building.