The agent platform of PACOD is pacod-ap. One deployment hosts any number of agents, each with its own knowledge base, reference facts, prompts, directives, settings, conversations and reviews. All of them share one sign-in through pacod-id, one admin console and one LLM gateway, pacod-gw. This post covers what an agent is, how a chat turn runs, how an agent is changed without a redeploy, and how it learns from its own traffic.

Built for answers that must be right

pacod-ap is built for agents whose job is to give correct, sourced answers in one domain, where a wrong price, rate or due date is worse than no answer. A help desk that quotes last year's price, or a tax assistant that misremembers a due date, cannot afford to sound merely plausible. Three rules follow:

  • Every number comes from a structured source of truth, never from the model's memory.
  • Every claim cites a source: a knowledge-base passage, a fact's source address, or an allow-listed web page.
  • Admins correct an agent without a redeploy. Models, prompts, directives, web search and caching apply on the next turn.

An agent is a profile

An agent is a profile: one row in the profiles table plus everything scoped to it. Two ship with the platform. starter, the default, is the help desk of a fictional note-taking service, "Nimbus Notes", with 29 reference facts, a small corpus in Bulgarian and English and a golden set of test questions. It is meant to be copied. taxes is an advanced example for Bulgarian tax and accounting, with its own prompts, facts, web-search allow-list and law corpus. That corpus is there to show retrieval on a real body of law. It is not kept up to date, and nothing the agent says is legal or tax advice.

Isolation between agents is a security boundary, not a convenience. Every per-agent row carries a profile_id, each agent has its own Qdrant collections, and every per-agent cache key carries the agent's id. People see only the agents assigned to them, and only those released to production. Admins work across all agents at once, or scoped to one with a selector at the top of the console.

How a chat turn runs

The guards run before anything streams, so a rejection is a proper HTTP status, not an error in the middle of a stream. The pacod-id token is verified, the agent must be assigned to the person, the language is detected and checked against the agent's languages, the gateway's daily token limit and a per-minute throttle apply, and a cheap classifier turns off-topic questions away or asks a clarifying question back. A fixed filter of illegal phrases runs before the classifier, so such a request never reaches a model.

Then a small LangGraph graph runs:

  1. plan: the planner returns JSON, naming the tools to call and whether the question is complex.
  2. execute tools: the planned calls run concurrently: kb_search; get_fact and list_facts over the Model Context Protocol; and web_search.
  3. synthesize: the primary model streams the answer from that context, with extended thinking on complex questions, and three suggested follow-ups come after it.

The planner picks a source by the kind of claim. Numbers, prices, rates, limits and dates come only from reference facts: typed rows in PostgreSQL, one generic shape for every domain. A help desk's plan prices and a tax agent's due dates share the same table, so a new domain is data, not schema:

{"fact_key": "plan_price_monthly", "variant": "plus", "category": "pricing",
 "legal_year": 2026, "value_type": "money", "value_num": 6, "unit": "EUR",
 "source_url": "https://nimbus-notes.example/pricing"}

The answer cites the fact's source_url. When a person asks for a number explicitly, the planner also searches the knowledge base, and the web where it is allowed. The fact wins, and if the sources disagree, the answer says so.

A table of reference facts in the pacod-ap console, such as a 30-day data retention after cancellation, a 12-character minimum password, a 14-day refund window and file-size limits of 25 MB, 250 MB and 1 GB per plan, each with its category, the year 2026 and a link to its source.
The starter agent's reference facts: a key, a variant, a category, a reference year, a typed value and the source an answer cites. The agent quotes these figures instead of recalling them.

The model never chooses whose data it reads. The code authorizes the agent before any work starts and sets the agent and the reference year on every fact lookup itself, so the planner can name a fact but cannot reach another agent's.

Explanations come from the knowledge base: bge-m3 dense search in Qdrant, a cross-encoder rerank, then small chunks expanded to their parent passage. A separate, down-weighted curated collection holds notes that admins wrote, promoted from a conversation or approved from the review queue. Recent information comes from web search, always together with the knowledge base, and limited to a per-agent domain allow-list.

The same agents are served over an OpenAI-compatible /v1, on the internal network, where each agent appears as a model. It runs the same guards and the same graph.

Changing an agent without a redeploy

Every setting that shapes an agent is a row keyed by the agent, read through a short Redis cache and falling back to the environment's default: the model for each role, the prompts, the directives, the web-search allow-list, the answer cache. A change applies on the next turn.

For larger changes there is the visual agent editor. An agent is a graph of components on a canvas: the guards, the answer cache, the conversation memory, the planner, the three tools, the synthesizer, an optional verifier and the follow-up suggestions. So far every graph follows one template, the conversational RAG agent, so components are switched on or off and configured rather than wired freely. Each setting either inherits the agent's current value or overrides it.

A change is saved as a draft, deployed as a dev version that only admins can chat with, and published when it is ready. Versions are insert-only, so publishing and rolling back only move a pointer. An AI assistant drafts prompts and other texts from an instruction, but never touches models, numbers or switches, and nothing changes until the author applies what it proposes.

The Agent editor page of the pacod-ap admin console with one row, the Starter Agent, a conversational RAG agent in production with a changed draft, and buttons for the AI assistant, importing, managing all agents in the editor and creating a new agent.
The agent editor's list: each agent with its type, its state and its production, dev and draft versions.

The improvement loop

Each agent improves from its own traffic. A worker reviews an agent's recent conversations, redacted first, when an admin presses Run now, every day once the schedule is switched on, or for one turn picked from the Conversations page. A cheap model first decides which borderline turns deserve the expensive analysis; a turn the person rated is always analysed.

The analysis judges each answer against what the assistant actually read: the verbatim context of the answer, the directives in force, the conversation so far, the tool calls, and the passages retrieval found but discarded. That last part is what separates "the knowledge base had it and the ranking missed it" from "the knowledge base never had it". Two independent analysts and a judge reach the verdict, or one analyst and a critic; a panel that is not really independent says so.

Each finding goes where it can be fixed, with the fix written out:

Failure Where it goes
Content gap A knowledge-base candidate in the Review queue, for a human to edit and approve
Retrieval miss The turn's retrieval trace, for tuning
Hallucination, tone, behaviour A corrected answer and a proposed rule, which becomes an inactive directive on request
A clean answer A golden test, with the facts the answer must contain

Similar findings are grouped into recurring problems, and a recurring problem is the ticket a reviewer works on: forty turns that fail for want of one document are meant to make one ticket, not forty rows. Nothing reaches the curated collection until an admin approves it, unless an auto-insert flag, off by default, is switched on and strict checks pass; a claim the sources contradict is never inserted. Reviewers mark each finding useful, dismissed, duplicate or fixed, and the dismissal rate is the loop's own measure of whether it is getting better.

The dark Review process page of the pacod-ap console with a Run now button and options for a dry run, a forced re-analysis and dismissed turns, then the configuration, with a daily schedule at 02:00 UTC switched off, a 24-hour look-back, a concurrency of 4, the triage and analysis modes and a note on the judge's independence.
The review process: a run by hand or on a daily schedule, a cheap model for triage, and two analysts and a judge for the verdict.

Try it

On a laptop, the gateway answers with its mock providers, so a turn costs nothing:

make secrets-init
make up-all
make bootstrap-local
make smoke-all
open http://localhost:8080

make bootstrap-local loads the starter agent's facts and corpus, and make smoke-all runs one chat turn through the gateway and checks that the gateway's trace of it costs what the turn reports.

A few things are worth knowing before you rely on it. The move from a LiteLLM proxy to pacod-gw was checked against the gateway's mock, and the switch to real providers is prepared but has not yet been run with real keys. Agents speak Bulgarian and English; the interface also speaks German and Russian.

The product page has the containers, the security model, the agent editor and what has been tested. The code, the runbook and the walkthrough for building your own agent are in the repository. How the gateway handles each call is in the post on pacod-gw, and how people sign in is in the post on pacod-id. The live demo runs the starter agent, with its chat and its admin console.