How We Put an Entire Back Office Behind One WhatsApp Number

The problem

Every operational fact the company owned started life as a WhatsApp message. A site engineer sent a voice note about a delay. A supervisor sent twelve photos. A manager asked, in a group of thirty people, who was covering tomorrow.

Three consequences followed, and every operations manager reading this will recognise all of them:

• Nothing was captured. A commitment made in a group chat existed only in that group chat. It reached a task list only if somebody chose to retype it.
• Nothing was searchable. "What did we agree about the third-floor handover?" meant scrolling.
• Reporting was manual. Assembling a weekly status meant one person reading chats and rebuilding the week by hand.

The obvious fix — move everyone to a project-management tool — had already been tried, by them and by nearly every company like them. Adoption fails because WhatsApp is where the work already is. So the brief became: leave the interface alone, put the system behind it.

The architecture

One WhatsApp number, several specialist agents, one orchestrator deciding who handles what.

• WhatsApp gateway (WAHA). Self-hosted, connected to the company's existing number. Every inbound message — text, voice note, image, document — arrives as a webhook.
• Backend service. Normalises the message, resolves the sender to a known person and project, and hands it to the orchestrator. Message state is tracked here so nothing is processed twice and nothing is lost on restart.
• Orchestrator agent. Classifies intent and routes. It does not do the work itself — that single decision is what kept the system debuggable as agents were added.
• Specialist agents.
  • Task capture — turns a message or voice note into a structured task with owner, project and due date.
  • Reporting — assembles status from the stored records on demand or on a schedule.
  • Media — handles inbound photos: attaches them to the right project, tags and stores them.
  • Document search — answers questions from the company's own documents and past decisions.
• Memory (Firestore). Conversations, entities, tasks and project state in a real datastore. Context is retrieved per request rather than crammed into a prompt.
• Models (Gemini via Vertex AI). Cheap models for classification and transcription, stronger models for report writing and judgement.

The human gate

The system drafts. A person approves. Every report, every summary and every outbound message goes to the operator on WhatsApp first, and it is sent only after a one-tap approval.

We expected to relax this once trust was established. We did not, and would not now. The approval step costs the operator a couple of seconds and it is the entire reason the platform was allowed to keep running: the person accountable for what the company says is still the person who says it.

The weekly report that nobody has to write

Once operational data was being captured as a by-product of normal chat, the reporting agent could assemble a weekly executive summary from records rather than from memory — what moved, what stalled, what is open and who owns it — and deliver it to the decision-maker on WhatsApp on a schedule.

This turned out to be the feature that sold the platform internally. Not the AI: the fact that the report arrived without anyone spending Thursday night building it.

What broke, and how we fixed it

This is the part most case studies leave out, and it is the part that matters if you are evaluating whether to build something like this.

WhatsApp delivery and session reliability

A self-hosted gateway is not a managed API. Sessions drop, the linked device disconnects, the container restarts. Early on, a dropped session meant messages were silently missed — the worst possible failure for a system whose entire promise is capture.

Fix: health checks on the session with automatic reconnection, a persistent inbound queue so messages survive a restart, and an alert to a human when the gateway has been down for more than a short window. Undelivered outbound messages are retried with backoff and surfaced rather than dropped.

Message-state tracking and duplicates

Webhook retries and reconnections meant the same message could arrive more than once, producing duplicate tasks — which destroys trust faster than missing a message does.

Fix: every message is stored with its platform identifier and a processing state before any agent sees it. Processing is idempotent and keyed on that identifier, so a replay is a no-op. The state machine also made "what happened to this message?" a query instead of an investigation.

Voice notes and dialect

Transcription trained on formal Arabic performed poorly on site: dialect, background noise, mixed Arabic and English, and technical vocabulary.

Fix: better transcription models plus a domain glossary, and — critically — the task-capture agent echoes back what it understood before creating the task. When it is wrong, the human corrects it in one message rather than discovering the error a week later.

Over-eager agents

The first orchestrator responded to too much. In a busy group chat, an agent that answers everything is noise, and people leave the group.

Fix: explicit addressing rules and a confidence threshold. Silence became the default behaviour, and the assistant now stays out of the way unless it is spoken to or has something scheduled to deliver.

Where it stands

The platform runs in production on the company's own WhatsApp number, with several specialist agents live and the weekly executive report going out on schedule.

What we can state plainly: the company did not change how it communicates, operational facts are now captured as a by-product of normal chat rather than retyped by hand, the reports arrive on their own, and the approval gate is still in place by choice rather than by necessity.

Detailed performance figures available on request.

Stack

WAHA (self-hosted WhatsApp gateway) · Node.js/TypeScript backend · n8n for scheduled and integration workflows · Firestore for memory and state · Google Vertex AI (Gemini) for classification, transcription and generation · Cloud Run for hosting · WhatsApp-native approval interface.