How I built a governed agentic platform across sales, engineering, manufacturing, stock, procurement and finance, on an ERP with no API
0. The mistake people make about this platform
When I describe it as "a budget modeller", people picture a spreadsheet with charts. That was true for about three weeks in December 2025.
What it became is the system my business runs on. In one session a user can ask what did we sell, who owns that account and are they on plan, can we build it, do we have the parts, did the supplier price move, will we be late and pay a penalty, what did engineering ship, what does the month-end pack say. Every answer is drawn from the same data, the same engine and the same permission model.
A finance dashboard reports the score. An operating system also reports why the score is what it is, who is accountable for it, and what can still be changed.
The main SAP Live Insights Dashboard is the landing page. At a glance, I can filter several views to inspect the business at different lenses. This is the main page for tracking critical KPI signals.
1. The constraint
SAP ECC 6.0. No HANA, no Fiori, no OData. "Reporting" means running a transaction code, exporting an ALV grid, emailing the file.
S/4HANA migrations start around $250K and run into millions; Gartner expects ~17,000 of 35,000 ECC customers to still be on legacy past the 2027 support deadline. We're one of them.
How do you get AI-native decision support out of an ERP with no API, without a
transformation budget, and without ever being wrong about a number? Alas, there is a manual step of running SAP exports, that are the ingested via an ETL workflow in the administration console. Every Friday, around close of business, I run the weekly exports for the SAP connectors.
The last clause on AI being grounded on truth data, is why this is mostly an architecture problem. An AI that is occasionally wrong about money is wrong fluently, at scale, and with convincing citations.
| Dimension | Value |
|---|---|
| Commits | 1,184 (Dec 2025 → Aug 2026) · one engineer |
| Routed views | 24 |
| AI tool definitions | 83, across 13 tool groups |
| Agent capabilities | 75 |
| Insight catalog entries | 170 |
| SAP connector families | 10 |
| RBAC roles / permissions | 12 / 34 |
| Test files | 455 |
| Monthly database spend | single digits |
Part I — Sales Command Center
The most-used surface in the platform, tracking performance of the sales account management team.
2. A control surface for account management
The organising idea is a single filter bar that re-scopes the entire page:
ACCOUNT MANAGER · REGION GROUP · REGION · SPECIFIC CUSTOMER [Reset]
│
└─ every KPI, chart, table and drill-down below re-computes
against the selected scopeFour hero KPIs (Total Portfolio Budget, Actual for the full year, Full-Year Variance, Open Order Book) sit above seven analytical sections, all of them scope-aware.
The organisational consequence is bigger than the feature. The GM (me) and the account manager look at the same page, not two reports that disagree. A portfolio review stops being "send me your numbers" and becomes "let's both look at the same scope".
Nothing was rebuilt for the account-manager view. It is the same components under a narrower scope, so a manager's numbers cannot diverge from management's.
3. The working list
Status badges — Behind Plan · At Risk · Exceeded · Unplanned — beside Planned Margin vs Actual Margin, full-year budget, actual, three years of history, open order book and variance. Filterable by plan status and customer type. Copy / CSV / Excel.
Two design decisions separate this from an ordinary table:
UNPLANNED is a first-class status. Revenue from a customer who was never in the plan is a fundamentally different conversation from a customer who beat their target. A single variance column would conflate the two, so the table reports them separately.
"Planned Margin %" is explicitly labelled as the budget-model weighted margin — not an SAP gross margin. The column header says so. Two quantities that share a name will eventually be compared, and the comparison will be wrong. The correction belongs in the column header rather than a footnote.
4. The same portfolio, four ways
FULL YEAR / QUARTERLY / 12 MONTHS changes the attainment question. Net Sales / Gross Profit / Quantity changes what performance means. A rep can hit revenue and miss margin; in a hardware business, revenue can hold while unit volume falls because price moved. Only the quantity lens exposes that.
5. The Explorer — self-service without a BI licence
Group by sales group · region · account manager · customer type · plan status · health status · customer. Split by customer type · plan status · health status · region. Metric variance · projected variance · actual YTD · YTD budget · open order book · customer count · pacing %. Sort worst / best variance, highest / lowest metric. Rows top 8/12/20/all.
Plus saved presets: Regional Gap · Account Watchlist · Order Book Cover · Unplanned Revenue · Channel vs Municipal.
This is how a sales review is run: group by manager, sort by worst variance, and work down the list. The product carries the caption "SAP actuals/order book stay separate from budget-model targets" because the two are never silently merged. That separation is what gives the variance figure meaning.
6. Three levels, one surface
Portfolio → customer → individual open order lines, without leaving the page or changing tool. One permission model the whole way down: an account manager sees their own accounts' lines, and the ABAC check happens server-side regardless of what the UI shows.
7. Every widget can explain itself
Every widget carries an Explain affordance that opens this drawer:
- What this shows — in business language.
- Data sources — with the exact ETL
defType(sap-sales-insights,sap-orderbook-insights, Budget Model). - How it is calculated — the actual formula and its assumptions.
- "Ask AI About This" — hands the widget's context straight to the copilot.
This is the 170-entry semantic layer made visible. It answers "where does this number come from" without a chat message or email, and it is the same catalog the AI reads through get_insight_definition. Because a single definition serves both the drawer and the copilot, the two explanations cannot drift apart. When creating an app that displays financial data, it is important to help your users along by providing helpful information - the app's intent was to be as self-explanatory to the user as possible.
Part II — The rest of the operating model
8. Product engineering
9. Manufacturing production
Public holidays are modelled as toggleable: "work this day" is a real scenario, not a calendar assumption. And scenarios are pinned and compared, because the useful artefact of capacity planning is the delta between two plans.
10. Stock — ship, build, unblock
Answers to four decision questions rather than an inventory listing. Note the honesty: v1 answers are explicitly gross_buildable, not MD04-reservation-aware free stock, and the UI says so. A number with a stated basis is worth ten without one.
11. Procurement
The scope filter is a semantic decision made in the ETL, not a UI convenience: product materials, product-adjacent, business overhead and unclassified spend are different signals, and mixing them makes every ratio meaningless. And promoting data quality to a hero KPI is the opposite of what most dashboards do — the honest answer to "how much are we committed to" depends on how much of the PO book could be classified at all.
12. Contracts
Built from a formal URS (User Requirements Specification) provided by a key account manager, whose backlog says an item is DONE only after production verification. One gap shows the rigour: the dashboard counted PO lines while labelling them POs — 528 lines against 499 distinct PO documents, with penalties counted per PO. The KPI block was silently mixing two grains.
13. FinPack — the official monthly pack
Income statement · balance sheet · cash flow · forecast drift · reconciliation, month by month. Declared authoritative for PBT in the shared grounding, so neither copilot nor agent may quote PBT from anywhere else.
Part III — The SAP bridge
importStatus === 'SUCCESS'
&& latestPointerUpdated === true
&& reconciliation.isBalanced === true // sourceRows === includedRows + excludedRowsIf one row disappears between spreadsheet and summary, the run fails and the previous good data stays live.
Part IV — The AI engine
14. Tools, not retrieval
This is a database-query problem, not a document-search problem. Retrieval's characteristic failure is a confident answer drawn from approximately the right rows.
Routing has two layers, deliberately not interchangeable: the chat profile selects the model, and the tool profile selects which subset of the 83 tools is exposed. Both operate under hard budgets of 15 tool calls, 10 iterations, a 300-second orchestration ceiling and 60 seconds per provider call.
15. The clarification gate
| Resolver outcome | Action |
|---|---|
| not resolved | clarify — stop and ask |
| confidence < 0.70 | clarify — catches all-token guesses |
| 0.70 – 0.90 | echo — "reading X as Y — correct me" |
| ≥ 0.90 | answer — proceed silently |
Where the reference is ambiguous, the handler returns candidates and no data. The comment explaining that decision generalises well beyond this codebase:
"The system prompt already instructs 'if ambiguous, ASK' and the model still guessed-and-proceeded. You cannot make the model ask by instructing it harder; you make answering impossible without asking by withholding the data."
To obtain a behaviour from a model, change what is possible rather than what is instructed.
16. Model auto-discovery and the health breaker
The platform is not pinned to a model string in an environment variable. It calls Anthropic's /v1/models on boot and every 12 hours, classifies by tier (fable → opus → sonnet → haiku), flags beta builds behind an admin opt-in, regex-validates ids, and persists a catalog to Cosmos so the admin dropdown renders without a browser ever calling Anthropic.
Failure posture is safe defaults, never throw: empty catalog falls back to the manual priority list; a failed refresh retains the stale catalog and surfaces the error to admins only.
Auto mode walks the tier order and manual mode follows an admin list. A demoted model moves to the back of the order rather than out of it, so the result is degradation rather than an outage. One resolver binds every Claude call: chat, Composer dry-runs and scheduled agents alike.
The health breaker takes observations from chat (which hits the provider orders of magnitude more than a weekly agent) and thresholds from admin policy. Agents consume its verdict but never write to it — one run is a sample size of one.
The source is candid about the limitation. A model matching no tier pattern lands in other, which auto mode never selects: "this is exactly how claude-fable-5 was missed."
Model choice is operational configuration rather than a deployment. When a new frontier model ships, an admin selects it from a dropdown that populated itself. When one degrades, the platform routes around it and recovers. Neither event requires an engineer.
17. Designed around the Claude Agent SDK
runtime.js is the only file importing the SDK. SDK types may not appear in stores, hooks, the tool server or ai-core; the SDK's tool() and createSdkMcpServer() are injected into the tool server. The cost of replacing the SDK is roughly a week spent rewiring that one file. Tests inject a fake async iterator, so the entire agent runtime is exercisable with no network and no SDK present.
const options = {
systemPrompt: composeAgentRunSystemPrompt({ agentSystemPrompt: def.systemPrompt }),
model: selectedModel, // admin policy → def.model → SDK default
disallowedTools: buildDisallowedToolsList(), // all 19 SDK built-ins
allowedTools: allowedMcpNames, // mcp__syntell__{logical}
mcpServers: { syntell: toolServerInstance },
permissionMode: 'dontAsk',
maxTurns: def.quotas?.maxTurns ?? 16,
abortController: sdkAbortController,
hooks: {
PreToolUse: [{ matcher: '.*', hooks: [policyGuard] }],
PostToolUse: [{ matcher: '.*', hooks: [citation, dataset, audit] }],
Stop: [{ matcher: '.*', hooks: [cost] }]
}
};Four SDK behaviours each cost me a day or more: tools: [] silently disables all tools including MCP ones — omit the key entirely; alwaysLoad: true is required or MCP tools hide behind built-in tool search; tool_response is a JSON string rather than an object; console.log from a hook never reaches container stdout.
One truth layer, two surfaces. The agent prompt is composed from the same exported blocks as the chat prompt, assembled at runtime and never persisted. Improving the grounding therefore requires no document migration, and drift becomes structurally impossible rather than a matter of discipline.
18. Two behaviours, and the bug they revealed
It refuses: SOURCE: none — tools unavailable, naming all three timed-out Jira queries. It separates fact from inference: "Facts above are tool-derived; the … points are my inferences/recommendations."
I re-ran that question for this edition and it timed out again, on three queries out of three. The degradation was better this time: it fell back to the Roadmap page KPIs, which carry the same live Jira data already resident in Cosmos, produced a quantified answer on throughput and work in progress, and refused only the parts it could not source.
So I read the code. Root cause: TOOL_TIMEOUT_MS = 5000 is the default. Production tools were recognised as slow and given a 30s override. No Jira tool was ever added to that map — so a cross-internet Atlassian call gets the same five seconds as an in-region Cosmos point read. Full write-up.
The fix is not a larger constant. Derive the timeout from a declared executionClass, then add a contract test asserting that every outbound non-Azure tool declares external-api. A silent default then becomes a build failure.
The pattern worth naming: beware the opt-in registry
Two of this platform's real defects share a shape: the missed model tier and the Jira timeout. Both were maps that new items had to be added to, where forgetting to add one failed silently. The remedy in each case is to derive the entry from a declared property, then test the derivation.
Part V — The agent platform
Five decisions: quarantine the SDK · lock the built-in tool surface · enforce policy in the handler, because the SDK can block a tool call but cannot filter its output · extract citations structurally from tool returns · validate before publishable, with failures to a Needs Review queue.
Chat-first authoring without a second chat runtime. The LLM has no publish tool. The invariant that made it shippable: V1-indistinguishability, enforced by a snapshot suite.
Ranking, thresholds, scope and metric-kind tagging are code. Identity resolution shipped first, and the ordering matters: a drifting customer key renders a rename as a removal followed by an addition, and the agent then narrates a churn event that never occurred.
Part VI — Governance
allow = hasPermission AND hasBuScope AND ownsResource. A decision oracle predicts the outcome and an executor issues the real request; any mismatch between the two is treated as a regression, and the comparison runs nightly.
Eleven admin capabilities: User Governance · Global Settings · Business Config & Admin · Feedback Triage · RBAC · AI Agents Policy · Audit Logs · AI Chat Logs · Operational Logs · ETL Pipeline · System Health.
Part VII — What this taught me
19. The competency map — and why it isn't all fives
Exposure to AI is cheap in 2026; every company has run a pilot. What is scarce is people who have carried an agentic system into production against real money and can say which decisions were load-bearing and which were fashion.
1 = aware · 3 = working · 5 = shipped and operating in production
| Core agentic engineering | Trust, safety & operations | Platform, data & delivery | |||
|---|---|---|---|---|---|
| Prompt & system-prompt architecture | 5 | Grounding, citations & provenance | 5 | Data engineering for AI | 5 |
| Tool / function-calling design | 5 | Security & authorization for AI | 5 | AI product design & explainability | 5 |
| Conversational chat engine | 5 | AI governance & guardrails | 4 | AI-augmented software delivery | 5 |
| Agent harness & orchestration | 5 | Model routing & multi-model ops | 4 | Cloud architecture (Azure) | 4 |
| Claude Agent SDK (production) | 5 | Cost control / FinOps for AI | 4 | RAG / vector retrieval | 2 |
| MCP — Model Context Protocol | 4 | Evals & output-quality measurement | 2 | Memory / long-horizon context | 2 |
| Multi-agent / durable workflows | 2 | AI observability (OTel gen_ai.*) | 2 | Fine-tuning / model training | 1 |
Why the low ones are low, stated plainly:
- Evals — 2. World-class contract tests; until recently zero quality evals. I built the hard part first and left the layer the industry standardised in 2025. It is the top item on my backlog and the first thing I would build somewhere new.
- Observability — 2. Solid bespoke correlation logging; no OpenTelemetry
gen_ai.*spans, so it interoperates with nothing. - RAG — 2. I deliberately refused RAG for structured financial data and can defend that in detail — but I won't claim depth I don't have. If your problem is genuinely document search, hire someone with a 5.
- Fine-tuning — 1. I have not trained a model. My leverage has been architecture around frontier models, not producing them.
A matrix with no low scores is a marketing document. These four are the gaps I would want to be hired to close.
20. Lessons
Build the calculation engine first, as a pure function.
Give agents tools, not documents.
Enforce policy where the data is, not where the prompt is.
You cannot make a model behave by instructing it harder — change what is possible.
Keep the deterministic layer deterministic.
Sequence the boring dependency first. Identity before Movement Pack; Movement Pack before agents.
Prove conservation before you publish.
A silent no-op is worse than a crash. The scheduler's findDue filtered on c.enabled = true — a field AgentDefinitionV1 does not have. The predicate never matched, so no scheduled agent run could ever fire, and nothing errored.
Beware the opt-in registry.
Write contracts, not prompts. A prompt expresses an intention; a contract is checkable, and only one of the two survives a refactor.
21. The honest scorecard and the backlog
A June 2026 maturity audit against ~60 primary sources, produced by a multi-agent workflow that adversarially verified every claimed gap, disproving 13 of 26. Verdict: Level 3 of 5, 4s on grounding, tooling and clarification; 2s on evals, observability, memory and durable execution. The sharpest finding was a recurring class of controls that were documented but not enforced.
3–4 on the hard, differentiating dimensions; 2 on the operational hygiene the industry standardised in 2025. Most enterprise teams are the mirror image.
Near: agent-output evals in CI · OpenTelemetry gen_ai.* spans · durable execution for the chat loop · enforce the declared controls · the tool executionClass fix. Medium: Composer V3 (artefact as viewport, lineage chip, plan-mode preview) · Material Cost Service · stock beyond gross_buildable · a memory layer. Long: Intersection Intelligence — a digital twin of every road junction in South Africa, shipped as a standalone service on its own plan so that, in the words of the design note, "if it does not work, it should be switched off without breaking the core of the app."
22. FAQs - Questions I've asked myself as I built it
"Why didn't you just use LangChain?"
The primary reason is that I set out to learn this from the ground up. I wanted to understand what a tool loop, a context window, a routing decision and a hallucination are, rather than learn a framework's abstraction over them. A framework would have defeated that goal.
There is also an architectural answer, and it stands on its own. The table below sets the suite against what this platform actually required:
| LangChain product | What it gives you | What I did instead | Verdict |
|---|---|---|---|
| LangChain (OSS framework) | Model/tool/prompt abstractions, chains, integrations across providers | Direct Anthropic Messages API + a frozen AiProviderTransport contract | Not needed. Its value is portability across many providers. I deliberately chose one. |
| LangGraph | Graph agent orchestration, checkpointing, durable execution, human-in-the-loop | Claude Agent SDK for the batch/agent path; a hand-rolled loop on the chat path | Competitive on the merits. LangGraph's checkpointing is better than my durability story today; see the gap described below. |
| LangSmith | Tracing, datasets, evals (incl. trajectory evals), prompt versioning, annotation queues | Bespoke correlationId event log; ~1,100+ deterministic contract tests; zero output-quality evals | The strongest case to adopt. This maps exactly onto my two lowest-scoring dimensions. |
| LangGraph Platform | Managed deployment, scaling, persistence for agent runs | Fastify service on an Azure App Service I already pay for | Not needed. I have one region, twelve users and an existing plan. |
| LangMem / memory | Long-term agent memory | No memory layer at all (scored 2/5) | Open. A real gap; not obviously best solved by a framework. |
The decisive point is that the agent loop was never the hard part. Anthropic's Agent SDK supplied the tool loop, sub-agents, hooks, permissions and session handling. The research note that locked in the decision records that the SDK "saves us from rebuilding the runtime / orchestration layer," reducing v0 from weeks of work to days.
The hard part was everything a framework cannot supply: ABAC enforced inside the tool handlers before data reaches the model; the Movement Pack, which pre-computes typed, ranked, content-addressed deltas so that the model narrates a result rather than computing one; a 170-entry governed insight catalog; and an SAP reconciliation gate that refuses to publish when row counts drift. LangChain has no opinion about any of it, because these are invariants specific to my business. Adopting a framework would have placed an abstraction over the tractable part of the problem and left the difficult part untouched.
Cost is a real constraint here rather than a rhetorical one. The platform runs on a small budget for roughly twelve internal users. LangSmith's Plus tier is $39 per seat per month plus trace overages, and a typical production pairing of LangSmith with LangGraph Platform lands between $175 and $375 a month before model spend. For a one-person team that is a material line item, and each dependency is also something that can break, change licence or reprice.
Where I would change my mind. My two weakest dimensions are evaluation and observability, both scored 2 out of 5, and both are LangSmith's core competence. The sensible response is not a framework rewrite but the adoption of LangSmith, promptfoo, Braintrust or DeepEval for the evaluation and tracing layer specifically, against a codebase whose interfaces are already frozen and contract-tested. The aim would be to close a named gap rather than to adopt a framework wholesale.
"Why go all-in on Claude? Why not build a provider-agnostic router?"
I considered it, and the architecture still carries the seam. AiProviderTransport is a frozen, asserted contract — the orchestration layer talks to a provider, not to Anthropic. providerCapabilities.js says so in its own header: adding OpenAI or Gemini "would add sibling files (openai.js, gemini.js) with the same classify() / resolveThinkingConfig() contract."
Precision matters here, because this is the point at which such claims are usually inflated. There is a seam, not a router: one implementation exists. The maturity audit scores the dimension 3 out of 5 and names the gap directly as "sophisticated single-provider routing; no cross-provider failover."
Stopping there was a resourcing decision rather than an architectural belief. When you are building at speed you choose a platform and proceed. I am one person running an entire business, and a provider-abstraction layer is a permanent tax on every feature, paid daily against a risk that has not yet materialised. I therefore integrated vertically on Anthropic: not only the LLM API but the Agent SDK, the MCP tool surface and the hook model. That verticality is what reduced the agent framework from months of work to days.
Inside that bet the platform does carry automatic failover: model auto-discovery every twelve hours against /v1/models, tier classification, and a health circuit breaker that demotes and recovers without intervention. The failover operates within the model family rather than across vendors. In a total Anthropic outage the AI features degrade while the rest of the platform does not, because every number on every screen originates in Cosmos and the model only narrates it.
I'd revisit on a hard trigger: a sustained availability problem, a pricing shift, or a capability that only exists elsewhere. The seam is there so that day is an implementation, not a rewrite.
"Why Cosmos DB and not a relational database?"
The choice began as a cost decision and survived for engineering reasons. What started as a proof of concept on a small budget grew into an enterprise application, and a document store, structured with discipline, turned out to fit the problem better than the relational default:
- Schema evolution without migrations. Fiscal-year structures, contract shapes and connector outputs changed repeatedly over eight months. Not one of those changes needed a migration window.
- The ETL has no impedance mismatch. SAP exports arrive as records and land as documents. A connector run produces artefacts and synchronises documents, with no ORM, no staging schema and no mapping layer between the export and the store. That accounts for much of why the ETL is ten connectors rather than a data-engineering department.
- The partition key is the security boundary. Everything partitions on
/pk, carrying business unit and fiscal year. The authorization rule ishasPermission AND hasBuScope AND ownsResource, and business-unit scope is enforced in the physical storage layout rather than in aWHEREclause that someone can forget to write. - Operational features I'd otherwise have built. TTL gives one-year audit retention as a field, not a cron job. ETags give optimistic concurrency without a locking strategy.
- One dependency, one region, an emulator for local dev.
npm run dev:upboots the emulator; there is no second database technology to run, secure, back up or patch.
The counterpoint is real: cross-container joins and aggregations are my problem rather than the engine's, and they are hand-written in the application layer. A relational database would make ad-hoc analytical queries considerably easier. That is also why the AI reads a governed semantic layer of roughly 170 curated insights instead of composing SQL. The storage decision and the tool-catalog decision follow from the same premise.
"Why a fixed tool catalog instead of text-to-SQL?"
Because a generated query cannot be authorized in advance. Each of the 83 tools enforces ABAC inside the handler, before a single row reaches the model, and returns the citations that the hooks accumulate. Text-to-SQL inverts that arrangement: the model composes the access path and the audit happens afterwards. Nor can a generated query be governed. dbt's 2026 benchmark makes the case for a semantic layer in preference to text-to-SQL, and the insight catalog is that layer, versioned with its formulas, assumptions and sources.
"How do you stop it inventing numbers?"
The mechanism is structural rather than instructional. Deterministic tool handlers emit the citations and chart datasets, PostToolUse hooks accumulate them, and the model is never the source of provenance. The Movement Pack pre-computes typed, ranked, ABAC-scoped deltas with content-addressed identifiers, so the model narrates a computed fact rather than performing arithmetic. Below a confidence threshold the clarification gate withholds the data outright instead of instructing the model to take more care.
One part of that apparatus does not yet work. The ClaimV1/FactV1 refine gate passes trivially, because a boilerplate fact is always attached. The audit caught it, it is documented, and it sits on the near-term list. A guarantee that always passes is not a guarantee.
"What's the weakest part of the platform?"
Durable execution. The dispatch queue is written but has no consumer, so a process restart orphans any in-flight run. The scheduler's stale-lease window of five minutes is shorter than the fifteen-minute run timeout, which leaves a latent double-fire. This is a documented Phase 4 deferral rather than an oversight, and the client mitigates it with auto-resume, but it remains the single largest reliability gap and the one place where LangGraph's checkpointing would have bought me something real.
"No evals? In a financial application?"
Correct, and it heads the backlog. What exists is the opposite trade: more than 1,100 deterministic contract tests, frozen versioned schemas, and a named regression test for every incident, including penalty-netting, the order-book double-count and the M20 cabinet. Every model call in every test is a deterministic fake. That yields strong guarantees about behaviour and none about answer quality. The probabilistic layer belongs on top of the deterministic gate rather than in place of it.
"How does one person keep this from rotting?"
A written contract carrying a pre-review self-check, each item of which exists because of a specific and embarrassing miss: assert the observable result rather than the intermediate you produced; diff both sides of every interface; run the full gate after touching a contract-tested artefact; enumerate every entry point; declare no silent caps; accept no prompt-only correctness. The recurring root cause it was written to eliminate is stopping at "my code does what I intended" instead of "the consumer gets the right result." Appendix A records that same lesson failing eight further times in a single afternoon.
"What would you do differently?"
Instrument with OpenTelemetry from the first day, because retrofitting gen_ai.* spans over a bespoke correlation scheme is pure rework. Build the evaluation harness before the twentieth feature rather than after it. And enforce a control before declaring it: the sharpest finding in the audit was a class of controls that were persisted yet inert, where an admin sets restrictedFields in the interface and believes it takes effect. Documentation describing runtime behaviour you have not written is worse than no documentation, for the same reason that a test agreeing with your bug is worse than no test.
The talk deck - If I were asked to give a talk at an AI conference
Ninety slides with speaker notes throughout, covering the same material as this post in a form built to be presented. It runs entirely in the browser and needs nothing installed. Press S for the speaker view with notes and a timer, and Esc for the slide overview.
Deck and figures are hosted at github.com/khanmjk/syntell-bizops-portfolio.
No comments:
Post a Comment