Air-gapped bilingual AI assistant for a government client

Context and constraints

A government client wanted a generative AI assistant its staff could use in English and Arabic. It had to handle chat, questions about their own documents, summaries, side-by-side comparison, translation, a searchable knowledge base, and voice in and out. All of it had to run on the client’s own servers, inside a fully offline network.

Those conditions drove most of the technical choices:

• The environment is fully offline. Every model, package and container image had to be available inside the network, at a version fixed in advance.
• The client set the baseline. Versions, ports and limits came from the client’s own specification, and the client’s infrastructure team owned the operating system and GPU driver layer.
• All the models had to fit a fixed GPU budget on NVIDIA H200 hardware: chat, Arabic generation, safety checks, speech and embeddings.
• Arabic had to work end to end, from reading PDFs through retrieval and prompts to speech.
• Safety checks run on every message and every reply, so every answer the assistant gives has passed them.

My role

We were a team of three, and I took on most of the technical work. I was on site with the client in September and October 2026. Requirements and design work started in June 2026. From September the work moved to hardening, deployment and rollout, which ended with the handover at the end of October.

The parts I owned:

• running the discovery sessions and drafting answers to the client’s open items
• co-writing the requirements and design documents
• converting the application to run fully offline, and the offline release tooling
• a security self-review before deployment, and fixing what it found
• the GPU benchmarks on the client’s server on 4 October 2026, and the Arabic model re-tune that followed
• the October release deployments
• the on-site code changes: the bilingual hybrid retrieval (dense and keyword search with reranking, Arabic stemming, cross-language lookup), guardrail upgrades, a scope gate that declines out-of-scope questions, and loading the knowledge base
• troubleshooting the rollout, from fixing reversed Arabic text in PDFs to resyncing a server clock

The application itself started from an earlier internal product, which the team adapted for this client. The client’s infrastructure team prepared the servers and owned the operating system and GPU driver layer. My work sat above that layer, in model serving, the application and its releases. The handover documents were a team effort.

Discovery to requirements

I ran the discovery sessions with the client. The client then raised a long list of open items from discovery, and later a further set of comments on the business requirements. I drafted the answers to both. Where a limit applied, the answer set it out in writing, so expectations were clear early. One example: replies could not be guaranteed to match every Arabic dialect.

I co-wrote the business requirements (BRD), solution design (SDD) and low-level design (LLD) documents and revised them through client and partner review rounds. Other teams wrote the infrastructure and DevOps sections.

Architecture

Sign-in Web app Orchestrator Guardrails Knowledge base Model serving Observability

Select a part of the diagram to see what it does and which part was mine, or trace one question through the system. Use the arrow keys to move between parts.

Trace a question

Sign-in goes through Keycloak, which connects read-only to the client’s directory, and staff use a React web app. Every question goes to the orchestrator, a FastAPI service, which works through three steps. It first checks a short list of approved answers: questions the organisation answers in its own fixed words. Next, a scope gate declines questions outside the assistant’s remit. Everything else goes to retrieval and generation.

Guardrails sit on both sides of the model. A fast CPU filter and a self-hosted safety model check every incoming message, and every block of a reply is checked as it streams out. If a check cannot run, the request is refused.

The models run on the client’s own GPUs. vLLM serves the chat, Arabic, safety and speech-to-text models, and dedicated servers handle embeddings and text-to-speech. The knowledge base is a Qdrant collection with hybrid dense and keyword search and a reranker. Prometheus and Grafana cover metrics and dashboards, logs are collected centrally, and Langfuse keeps a trace of every chat turn with the message content redacted.

Key decisions

Deployment and rollout

The first job was getting the application to run fully offline. I removed the hosted model path, pinned every version to the client’s baseline and built every in-house container image offline. Before deployment I ran a security self-review of the code and fixed every verified high-severity finding, adding regression tests for the backend and frontend fixes.

I ran multiple release cycles on the client’s servers in a short timeline in October 2026. Most release records include checksums and rollback steps. Findings from the client’s acceptance testing went into releases the same week, including fixes to retrieval depth and to how the assistant handles a question built on a wrong premise.

Rollout needed two fixes, and the benchmarks led to one tuning change.

Making Arabic PDF text searchable

Arabic questions were finding only English sources. I traced it to the PDF reader used when loading the knowledge base, which stored Arabic text with the letters reversed, so even common Arabic words never matched. I switched to a different reader and re-indexed the knowledge base.

Bringing a server clock back in sync

A read-only check found one server’s clock running far past the few seconds of drift that sign-in tokens tolerate, before any sign-in failed. I pointed time sync at the client’s time servers, which brought the clock back within tolerance.

Re-tuning the Arabic model

On 4 October 2026 I benchmarked the deployed models with vLLM’s serving benchmark. After the first round I changed the Arabic model’s KV-cache precision and raised its limit on concurrent sequences. At the same load, its output throughput rose by more than a quarter. Every benchmark run that day for the chat, Arabic and safety models finished with no failed requests.

Handover

On 28 October 2026 the final release was deployed and handed over for user acceptance testing, together with the team’s handover documents: architecture, API, security, deployment, administration and troubleshooting guides, and user guides in English and Arabic.

Results

Each result sits next to the conditions it was measured or delivered under, and each figure comes from a single run. Load tests used simulated users.

What I’d carry into the next engagement

• Build the evaluation question set before the first release, so every change, the Arabic re-tune included, is measured against the same yardstick from day one.
• Agree who owns each day-two task (monitoring, upgrades, data changes) in the first release plan.
• Use a release-record template with rollback steps built in.

How I worked

During development I used Claude Code as my main coding agent, under a written security plan for the engagement. A managed policy enforced the plan, and neither a user nor a repository could loosen it:

• Claude Code could read code and propose edits. Shell, web, MCP and plugin tools were removed, so I ran every command myself.
• Every edit came to me as a diff to approve, and the modes that skip approval were switched off.
• Everything in the project except source code was classed as restricted. Deny rules blocked credential files and common data, document, log and media file types, so none of it could enter a session.
• A firewall allowlist let it reach only Anthropic’s API and sign-in. Telemetry, feedback uploads and auto-updates were off, and the version was pinned.
• Session transcripts stayed on the machine as the audit record.

A project rulebook (a CLAUDE.md file) set the invariants, data-handling rules and release gates, and a scripted gate of type checks, linting and automated tests ran before every release build.