Skip to content

Cortex documentation ingestion runbook

What this powers

Traylinx's in-app assistant and Tytus documentation Q&A retrieve product knowledge from the same production Cortex corpus. The web chat calls https://api.makakoo.com/ma-cortex/v1/api/v1/chat with app_id=traylinx. The running service is Kubernetes cortex-ms in strato-prod/production; its source repository is traylinx/tytus-cortex, not the separate Cortex-platform project.

Knowledge lives in cortex_production.memories, scoped exactly to:

user_id = SYSTEM_TRAYLINX
app_id  = traylinx_ground_truth
embedding = vector(1024)

Personal memories, business configuration, organizations and delivery are outside this workflow. Publishing documentation to Netlify does not reindex Cortex. No Cortex/AgentRelay/Auth/Metrics/Kong runtime deployment or schema change is needed.

Sources and ownership

Source tag Source root Responsibility
docs-hub This repository's docs/ Public knowledge hub; registered app pages are generated mirrors
traylinx-user-manuals Traylinx docs/user-manuals/ Canonical app user documentation
traylinx-public-references Traylinx public/references/ Concise machine-readable product references

App repositories own product documentation. Edit the source, register new mirrors in _sources.yml, then run scripts/sync_sources.py --check and --apply after reviewing drift. Do not hand-edit registered mirrors or publish unrelated drift. Add new pages to mkdocs.yml; register public references in Traylinx public/llms.txt.

AgentRelay's owner/visitor guide is traylinx/docs/user-manuals/agent_relay.md, mirrored as docs/hubs/user-guide/features/agentrelay.md. Its public companion is traylinx/public/references/agentrelay.md. Keep the Projects/Assets and Studio guides consistent. Do not index entire service repositories, customer records, release logs, secrets or historical plans. Sprint specifications stay outside docs/.

Model and provider

Use mistral-embed, 1,024 dimensions, through the OpenAI-compatible SwitchAI gateway https://switchai.traylinx.com/v1/embeddings. Both running production Cortex pods were checked on 4 October 2026; their retrieval model was mistral-embed. SwitchAI's live model metadata reports owned_by=mistral: Mistral supplies the model; SwitchAI is the gateway, not OpenAI the model provider.

Keep indexing and retrieval on the same model. Override stale local .env values with EMBEDDING_MODEL=mistral-embed. Protect EMBEDDING_API_KEY; never print it, DSNs, cookies or tokens. The indexer loads scripts/.env; export mode does not connect to any database or require a local database.

Targeted update: normal path for new or changed documents

  1. Review the canonical docs, mirror drift, links and mkdocs build. Freeze the selected file contents; record per-file hashes, source revisions, model, dimension and target tags in a private manifest.
  2. Obtain approval for paid embeddings and the production update. Choose every affected file variant (canonical manual, hub mirror and public reference), not a whole repository. Existing rows are grouped by their exact four tags: ['docs', source, category, filename_stem].
  3. Run the existing indexer with explicit file paths, matching --source arguments and --dry-run. Review chunk count/content. Then export all embeddings before touching the database. Provider responses must contain one unique indexed vector per chunk and 1,024 finite numeric components.

The AgentRelay update used seven files and 69 chunks. A repeatable export example:

cd /Users/sebastian/Projects/makakoo/agents/traylinx-docs-hub
HUB="$PWD"
APP=/Users/sebastian/Projects/makakoo/front/web_apps/traylinx
OUT=/Users/sebastian/MAKAKOO/tmp/cortex-selected-docs.jsonl
umask 077
EMBEDDING_MODEL=mistral-embed python3 scripts/ingest_to_cortex.py \
  --export-embeddings "$OUT" \
  --docs-dir "$HUB/docs/hubs/user-guide/features/agentrelay.md" --source docs-hub \
  --docs-dir "$HUB/docs/hubs/user-guide/features/projects-and-assets.md" --source docs-hub \
  --docs-dir "$HUB/docs/hubs/user-guide/features/studio-tools.md" --source docs-hub \
  --docs-dir "$APP/docs/user-manuals/agent_relay.md" --source traylinx-user-manuals \
  --docs-dir "$APP/docs/user-manuals/03_projects_and_assets.md" --source traylinx-user-manuals \
  --docs-dir "$APP/docs/user-manuals/studio_tools.md" --source traylinx-user-manuals \
  --docs-dir "$APP/public/references/agentrelay.md" --source traylinx-public-references
shasum -a 256 "$OUT"

For a dry run, substitute --dry-run for --export-embeddings "$OUT". Choose a new output filename for each approved update. These are local file operations and paid embedding calls, not local indexing.

Production import

Use scripts/update_cortex_documents.py, a lightweight stdlib/asyncpg operator. Select a healthy ready Cortex pod after listing it with an explicit context:

kubectl --context strato-prod -n production get pods -l app=cortex-ms

Copy the approved artifact and operator to a new mode-0700 /tmp/ directory in that pod. Run the operator there with --artifact, --approved-sha256, --backup and --receipt. Both destination files must be new. Every kubectl command must explicitly name --context strato-prod -n production. The pod's existing DATABASE_URL and EMBEDDING_MODEL supply runtime settings; do not copy credentials or import the application/framework into the pod.

The operator validates the checksum, all vectors/tags, actual database, dimension and runtime model. In one transaction with a bounded lock it:

  • Saves all affected existing rows, including IDs/timestamps/tags/vectors, in a mode-0600 backup before deletion. Newly introduced groups are retained in the backup's target list even when they had no prior rows.
  • Deletes only those exact file groups within the shared documentation namespace.
  • Inserts the complete approved artifact and checks resulting counts.
  • Checks a count/content/vector/tag fingerprint of all unrelated shared docs. Any mismatch or insert failure rolls back the transaction.

Copy backup and receipt to a protected evidence directory outside repositories and indexed roots. The operator prints committed:true only after commit. If the later receipt-file write fails, inspect the committed output/database rather than blindly running the update again.

Verification and rollback

Check actual counts by source and selected tags, complete vectors and expected content. Counts change: use the current manifest/receipt, not a permanent fixed number. Then use authenticated /v1/docs/search with k=5 and no source filter for AgentRelay explanation, creation/linking, email/replies, attachments and receipt status, plus API keys, Tytus and billing regressions. Test English and German.

/v1/docs/answer can synthesize over the same corpus without creating chat history, but its prompt/ranking differs from chat. A fresh approved information-only Traylinx chat is the final integration check. Never request Activate, Verify, Save or Send as an inferred knowledge test. Record sources and answers separately; plausible model text alone does not prove the expected docs were retrieved.

For an approved rollback, copy the retained backup into a healthy pod, compute its checksum and run the same operator with --restore, that backup as --artifact, its hash as --approved-sha256, and new backup/receipt destinations. It restores the original rows/IDs/timestamps and removes newly added target rows; unrelated groups remain untouched. Validate provenance before rollback. Do not replay a stale backup over a later legitimate update without fresh approval.

Dated production receipts

Update Before Selected old rows Inserted After Other docs
14 May 2026 Tytus full refresh 621 Entire shared corpus 988 988 Historical full refresh
4 October 2026 AgentRelay/Assets targeted update 988 26 69 1,031 962 unchanged, fingerprint verified

The October artifact SHA-256 is 54a5c54f24a30a2aa7a0c8cb222d9f44ac30dc6b8cdc55d20e79bd46034729f9. Detailed private manifest, backup, production receipt and retrieval/chat evidence are in /Users/sebastian/MAKAKOO/data/reports/agentrelay-cortex-knowledge-2026-10-04/. These are not indexed support content.

Historical full-refresh path: not for partial documentation

The old ingest_to_cortex.py --mode refresh database path replaces the entire SYSTEM_TRAYLINX/traylinx_ground_truth corpus. Never run it with only AgentRelay or other selected files: it would remove unrelated knowledge. A future full refresh requires a separately reviewed complete three-root manifest, backup and approval. The new targeted operator is the normal path for document updates.

Adjacent assistant layers and limits

Frontend src/utils/agenticPrompt.js, FAQ suggestions and public references are separate layers. Changing a quick-action label does not add retrieval knowledge. The deployed chat schema currently does not consume frontend config.system_prompt; that issue is separate from indexing. Shared-doc temporal ranking in the chat path is also an existing follow-up, not changed here. Retrieval/answer latency includes gateway/model work and is not a measurement of AgentRelay's owner API performance.