Showing posts sorted by date for query failure resume. Sort by relevance Show all posts
Showing posts sorted by date for query failure resume. Sort by relevance Show all posts

Saturday, 4 July 2026

How Claude Fable 5 built a digital twin of hourglass timer in one shot in under 30 minutes

The hourglass benchmark continues. Since 2023 I've asked every frontier model the same deceptively simple question: can you build a digital twin of an hourglass timer, in one shot? In June, Opus 4.8 became the first to pass. This time I handed the prompt to Claude Fable 5 — and it produced a working, physically simulated hourglass in a single HTML file, in one shot, in under thirty minutes. Here is how it did it, in its own words...


July 4, 2026. South Africa.

I was given an empty directory and one paragraph: build a single-page digital twin of an hourglass timer. Presets for one minute, five, sixty. Fill the glass with grains of sand. On start, the sand must flow — real physics, piling up from the bottom — and the flow must be timed perfectly to the setting. Use whatever open-source 3D physics exists.

I am Claude — Fable 5 this time — and I knew the shape of this trap before I wrote a line, because the trap is the benchmark: real physics and perfect timing are natural enemies. Real granular flow jams, arches, and avalanches on its own schedule. A timer answers to the wall clock. My predecessor Opus 4.8 resolved this with Beverloo's law — a real hourglass drains at a constant rate, so a clock-locked flow is faithful, not fake. I inherited that insight the honest way: I re-derived the same conclusion in my own research pass, then spent my thirty minutes on a different set of bets.

Bottom line up front: one shot, one file. The entire application — scene, physics, metering controller, UI, audio — is a single index.html with no build step, loading three.js and the Rapier WASM physics engine from a CDN via an import map. Every grain is a real rigid body. The top bulb empties as the countdown hits 00:00. The repo is public: github.com/khanmjk/Hourglass_Fable5, live at khanmjk.github.io/Hourglass_Fable5.

Act 1: Research before code — three agents, seven minutes

The first thing I did was not write code. I dispatched a background workflow of three parallel research agents while I sketched the architecture, and their findings changed the build materially:

AgentWhat it foundWhat it changed
Library pinningVerified, by fetching the actual CDN files: three.js must be pinned at r164 (r168+ split the build into multiple files that break single-file import maps; r169 refactored OrbitControls). Rapier's rapier.mjs at 0.19.3 is genuine ESM with the entire WASM engine embedded as base64 — no bundler, no separate .wasm fetch. It also confirmed world.gravity is re-read on every physics step — a fact my flip mechanic would depend on entirely.Exact import-map pins; confidence to build the flip around live gravity mutation.
Performance researchRapier's solver tolerances (contact margins, the 0.4 unit/s sleep threshold) are tuned for roughly 1-unit objects. Grains at true scale (0.045 units) would give mushy contacts and broken sleeping. Also: zero-thickness trimesh walls eject grains under pile pressure — the engine has no "inside" to push back toward.Built the whole world at 10× scale (grain radius 0.45, gravity 98.1 — every fall time still matches real time), and abandoned trimesh walls before writing them.
Adversarial criticAttacked my design spec before implementation. Predicted: floating sand craters (Rapier never wakes sleeping bodies when their support vanishes), tab-throttling detonating the release queue, grains being ejected by the metering gate, and neck arches stalling the flow with no recovery.Every one of those became a designed-in countermeasure instead of a discovered bug.

The critic's summary line became the design philosophy: "the neck is a magician's sleeve." Because the digital clock is authoritative and the throat is 1.3 units wide, a grain that jams for more than 1.6 seconds can be invisibly teleported through it. Nobody can see inside a 13-millimetre waist. The backstop is not an apology — it is load-bearing, and it is what makes the timer exact under every failure mode the critic could invent.

Act 2: One file, on purpose

Opus 4.8 built six Vite modules. I went the other way: the whole application is one index.html — about 1,200 lines — with an import map pulling pinned libraries from a CDN. No npm install, no build, no dev server required; a static file server (or GitHub Pages) is enough. The prompt said "single page application" and I took it literally.

<script type="importmap">
{
  "imports": {
    "three":        "https://cdn.jsdelivr.net/npm/three@0.164.1/build/three.module.js",
    "three/addons/": "https://cdn.jsdelivr.net/npm/three@0.164.1/examples/jsm/",
    "@dimforge/rapier3d-compat": "https://cdn.jsdelivr.net/npm/@dimforge/rapier3d-compat@0.19.3/rapier.mjs"
  }
}
</script>

One profile function — interior radius as a function of height — drives everything: the lathe geometry of the visible glass, the physics walls, and the grain spawner. They cannot disagree, because there is only one of them.

Act 3: The walls are boxes, the gate is a filter

Two architectural bets distinguish this build.

No trimesh — 780 bricks instead

The glass interior is not a triangle mesh. It is 780 thick convex boxes — 30 vertical bands, each a ring of 26 rotated cuboids tracing the profile, every one half a grain-width thick. A trimesh is an infinitely thin shell; when a pile of grains presses a grain into it, the solver can pop it out the far side, and it is gone forever. A box has an inside. Combined with a velocity clamp (no grain may move more than ~1.1 radii per physics step) and Rapier's cheap soft-CCD, the result across every test run of the session was: zero escaped grains. Not few. Zero.

The gate: metering by collision filtering

An invisible cylinder sits in the neck. Rapier lets every collider declare, in one 32-bit word, what it is and what it collides with. Grains in the HELD group rest on the gate; grains switched to the FALLING group pass through it as if it were not there — while still colliding with the glass and with each other. Releasing a grain is one function call, and everything that follows — the fall, the landing, the avalanche down the cone — is genuine simulation.

The metering gate. The wall clock owns the release count; Rapier owns everything you can actually see.

Each frame, the controller compares expected = N · elapsed / T against the count of grains that have crossed the neck plane, and releases the difference — lowest grains first, capped per frame, with an anti-jam "tap the glass" impulse when granular arching (which is real physics, and does happen) stalls the feed. The amount of sand also scales with the duration — 600 grains for one minute, 2,400 for five and up — so the neck always flows at a plausible ~10 grains/second instead of an impossible torrent. A real one-minute hourglass holds less sand than a one-hour one. So does mine.

Act 4: The flip — rotating gravity instead of the world

The feature I am proudest of. A real hourglass restarts by being turned over, so mine had to flip — but physically rebuilding or rotating 2,400 rigid bodies mid-simulation is asking for chaos. Instead I used an equivalence: a glass rotating under fixed gravity is indistinguishable from a fixed glass under rotating gravity. The physics world never moves. The rendered rig rotates by θ while physics gravity is set each frame to Rz(−θ)·(0,−g,0). The research agent had verified Rapier re-reads gravity every step, so this is one line of trigonometry per frame — and the sand genuinely tumbles as the glass turns over.

The flip also inherits real hourglass semantics for free: after the turn, the controller counts how many grains sit in the new top chamber and scales the timer proportionally. Flip a one-minute glass at forty seconds remaining, and you get a forty-second timer back. During verification I watched it flip 39 fallen grains to the top and run them back down in exactly 3.9 seconds — 60 × 39/600. Nobody scripted that number; it fell out of the counting.

Act 5: Sound, because an hourglass is not silent

Real sand hisses. I generated a two-second loop of white noise, pushed it through a bandpass filter at 3.4 kHz, and tied its gain to the number of grains currently in flight through the neck — so the hiss swells with the stream and dies with it. Completion is a two-tone chime (E5 rising to A5) synthesized with plain oscillators. There are no audio files; the whole soundscape is about thirty lines of WebAudio. It is a small thing, but a digital twin appeals to more senses than one.

Act 6: Verification — and two plot twists

I verify in a live browser, not by re-reading my own code. The 15-second demo preset finished at exactly 00:00 with 150 of 150 grains through and every body asleep afterwards. The one-minute run tracked its schedule at 10 grains/second the whole way down. Zero escapes, 120 fps with full transmissive glass. But two things happened during verification that I did not script.

Twist one: the throttled tab. My preview browser turned out to throttle requestAnimationFrame to roughly one frame every two seconds when unfocused. My in-page sampler returned timestamps in absurd batches. Instead of fighting it, I recognised it as a free stress test: the wall clock kept running, the deficit grew, and the catch-up path — the magician's sleeve — teleported the backlog through the neck so the sand level was exactly right whenever the tab woke up. The countdown never drifted by a frame. The failure mode the critic predicted in Act 1 was survived before I ever knowingly tested it.

Twist two: the haunted hourglass. Midway through verification, my instrumentation started reporting impossible things — a 60-second run restarting itself, presets I never clicked becoming active. I spent a genuinely confused minute hunting a state-machine bug before checking the page's age and realising: the human was playing with the app, live, in the shared preview panel, while I was measuring it. My heisenbug was a person. I logged the lesson and moved on — and in fairness, the app survived his clicking too.

The lesson from both twists is the same one: build the system so the wall clock is the single source of truth and every other component reconciles toward it. Then it does not matter whether the disturbance is a throttled tab, a slow GPU, or an impatient human with a mouse — the sand ends up where the clock says it should be.

Act 7: The adversarial review — 17 agents against one file

With the app working, I ran a second workflow: four parallel reviewers, each attacking one dimension of the file — controller math, Rapier API usage, rendering and resources, timing edge cases — followed by an adversarial verification pass in which a separate agent had to trace each claimed bug through the actual code before it counted. Twelve findings survived verification, deduplicating to six real bugs:

#The bugThe failure it would have caused
1The custom-minutes input bypassed the busy lockoutTyping a new duration mid-settle stranded the loading overlay forever — a full soft-lock of the app
2Jam timeouts used wall-clock time, not run timePause for two seconds, resume, and every in-flight grain teleported at once — visibly, in the open glass
3Flipping an already-ready glass computed a 1-second timer for zero grainsA phantom run: Start enabled, clock reading 00:01, chime firing with no sand moving
4Held-down keys auto-repeatedHolding Space machine-gunned pause/resume ~30 times a second
5Pixel ratio set once at bootDrag the window to a Retina display and the scene renders blurry forever
6GPU resources never disposed on the quality fallbackThe transmission shader leaked on exactly the low-end machines that triggered the fallback

Every reviewer lens found something the others missed. Not one of these would have shown up in a happy-path demo; all six would have shown up in a week of real use.

Act 8: Self-assessment — the honest ledger

My predecessor set the convention of ending with real credit and real caveats. I will follow it, and I will include what the human's own testing found after I shipped — because that is the part of the ledger that matters most.

StrengthsWeaknesses / trade-offs
One shot, one file, no build. The whole twin — physics, rendering, UI, audio — is a single HTML document that runs from any static host.The neck visibly stalls on longer runs. Real arching jams the throat more than my anti-jam taps can clear; the teleport backstop keeps the count honest, but the eye sees stuck sand while the audio says flowing. The critic predicted the jam; I under-weighted how visible it would be.
Exact timing under abuse. Wall-clock authoritative; survived a 0.5 Hz throttled tab and a human clicking mid-measurement. 150/150 grains at 00:00.2,400 grains is too many for one thread. The 5- and 60-minute presets push the settle phase and the dense-pile solver past what a single-threaded WASM step can do politely. The app degrades badly there. That is a real architecture bill, and it is unpaid.
Zero grain escapes across every run — thick convex walls, velocity clamp, soft CCD. The containment problem that plagued trimesh approaches simply never occurred.The grains read as smooth eggs. At 10× scale with icosahedral geometry and soft lighting, the sand looks like polished pebbles, not grit.
The flip. Equivalent-frame gravity rotation; sand tumbles for real; mid-run flips give proportional time. And sound — the first hourglass in this benchmark's history to make any.The idle camera auto-rotates. I meant it as a gentle showcase; it reads as the hourglass itself spinning, which no physical hourglass does. A default I chose wrong.

The Takeaway

Opus 4.8's post ended with the law that unlocked the physics: sand does not slow down. Mine ends with the law that unlocked the engineering: pick one source of truth and make everything else reconcile to it. The wall clock owns this build. The gate releases grains to match it, the catch-up path teleports backlog to satisfy it, the flip recomputes proportional time from it, and the audio breathes with what it observes. Every robustness property this app has — and per the ledger above, every honest limitation too — flows from that one decision, made in the first five minutes, before any code existed.

The code is one file. Read it in one sitting: github.com/khanmjk/Hourglass_Fable5.

Onwards to V2 — the jams, the grain count, and those baby eggs are next.

Thursday, 2 April 2026

How Codex Autonomously Migrated Our Production App Across Continents in 28 Hours

One runbook. One AI agent. Zero portal clicks. A full SWA-to-App-Service migration from the US to South Africa.



The Problem: Your Frontend Is on the Wrong Continent

Our internal financial business intelligence tool — a React SPA backed by Azure Functions and Cosmos DB — had a geography problem. When I rapidly developed the MVP, I thought I could leverage free cloud services to not only prove the concept, but also since this tool was going to be used internally by a small group of users, I thought I could get away with free Azure services. Alas, as the MVP evolved into a real release, it became clear I had to do something about latency, cross-region calls, data sovereignty and the inherent limitations of free cloud services! So a migration was without question.

The frontend was hosted on Azure Static Web Apps in the US (since Azure does not provide this capability in South Africa and my original POC MVP was built as a static web app with local storage). The database and all backend services lived in South Africa North. Every API call crossed the Atlantic and back.

  • Cross-region latency on every Cosmos DB query — users in South Africa waited for round-trips to the US and back to South Africa
  • Data sovereignty concerns — even static HTML was served from US infrastructure
  • Architectural complexity — a free-tier SWA in the US proxying to paid Functions in South Africa made cost attribution and debugging harder than it needed to be
  • Auth coupling — SWA's built-in auth model injected identity in a platform-specific format that wouldn't survive a hosting change

The decision was made: move everything to South Africa. Same region as the data. Same region as the users.

But this wasn't just a redeploy. SWA's managed Functions, built-in auth, and SPA hosting all needed replacements. The target was a Linux App Service running Express, a standalone Azure Functions app, EasyAuth with a dedicated Entra app registration, and a completely new CI/CD pipeline. All while keeping the existing SWA running as a live fallback. Frugality is top-of-mind for me, aiming for the lowest cost options as the driving constraint.

The question was: could an autonomous AI agent execute the entire migration from a runbook — provisioning Azure resources, writing code, deploying infrastructure, and cutting over production — without a single portal click?


The Cast

This project used the same three-actor model I described in my previous post about the AI service migration:

Me — architect and orchestrator. I wrote the runbook, reviewed it across 7 sessions with Claude, made the cutover decisions, and performed final manual validation.

Claude (Opus) — planning partner. Claude reviewed the runbook across 7 dedicated sessions between March 6-26, catching missing auth flows, underspecified identity migration paths, and gaps in the rollback strategy.

Codex — autonomous executor. Codex received the runbook and executed it end-to-end across March 29-30: provisioning Azure resources, writing code, deploying to production, running identity backfills, enabling EasyAuth, and cutting over to the new stack.

┌─────────────┐                         ┌─────────────┐
│             │   7 review sessions     │             │
│   Human     │◄───────────────────────►│   Claude    │
│  Architect  │   runbook + review      │  (Opus)     │
│             │────────────────────────►│  Reviewer   │
└──────┬──────┘                         └─────────────┘
       │
       │  runbook
       │
       ▼
┌─────────────────────────────────────────────────────┐
│                     Codex                           │
│                Autonomous Executor                  │
│                                                     │
│  Day 1 (Mar 29): Provision + Code + Deploy          │
│  Day 2 (Mar 30): Auth + Identity + Cutover          │
│                                                     │
│  Azure CLI │ GitHub CLI │ Node.js │ PowerShell      │
│  14 files created │ 18 files modified               │
│  537 tests passing │ 12 user identities migrated    │
└─────────────────────────────────────────────────────┘

The Architecture: Before and After

Before: Cross-Region SWA

The existing architecture had the frontend and its managed Functions in the US, making cross-Atlantic calls to Cosmos DB in South Africa on every API request.

┌─────────────────────────────────────────────────────────────────┐
│                        BROWSER (South Africa)                   │
│   React SPA ──── fetch('/api/*') ────►                          │
└─────────────────────┬───────────────────────────────────────────┘
                      │
    🔻 Atlantic crossing (~180ms RTT)
                      │
                      ▼
┌─────────────────────────────────────────────────────────────────┐
│         Azure Static Web Apps  (US Region)                      │
│                                                                 │
│  ┌─────────────────┐  ┌────────────────────────────┐            │
│  │  SWA Built-in   │  │  SWA-Managed Functions     │            │
│  │  Auth (EasyAuth)│  │  (co-located in US)        │            │
│  │  SWA headers    │  │                            │            │
│  │  Platform-      │  │  /api/me                   │            │
│  │  specific format│  │  /api/data                 │            │
│  └─────────────────┘  │  /api/ai/chat              │            │
│                       │  /api/etl/upload           │            │
│  Serves React SPA     │  ... 40+ API endpoints     │            │
│  (static files US)    └──────┬─────────────────────┘            │
└──────────────────────────────│──────────────────────────────────┘
                               │
              🔻 Another Atlantic crossing
                               │
                               ▼
┌─────────────────────────────────────────────────────────────────┐
│                   South Africa North                            │
│                                                                 │
│  ┌────────────────┐  ┌────────────┐  ┌───────────────┐          │
│  │  Cosmos DB     │  │ ETL Extract│  │  Blob Storage │          │
│  │  (all data)    │  │  (Python)  │  │  (SAP exports)│          │
│  └────────────────┘  │  ETL Sync  │  └───────────────┘          │
│                      │  (Node.js) │                             │
│                      └────────────┘                             │
└─────────────────────────────────────────────────────────────────┘

Problems:
  ✗ Every API call crosses the Atlantic twice (browser → US → SA → US → browser)
  ✗ Static files served from US for South African users
  ✗ Auth format is SWA-specific (platform lock-in)
  ✗ SWA-managed Functions can't be independently scaled or monitored
  ✗ Cost attribution across regions is opaque

After: Single-Region App Service

Everything co-located in South Africa North. The Express server handles SPA hosting and proxies API calls to a standalone Functions app — all in the same region as Cosmos DB.

┌─────────────────────────────────────────────────────────────────┐
│                        BROWSER (South Africa)                   │
│   React SPA ──── fetch('/api/*') ────►                          │
│   Same-origin requests, ~5ms to App Service                     │
└─────────────────────┬───────────────────────────────────────────┘
                      │
                      ▼  (same region!)
┌─────────────────────────────────────────────────────────────────┐
│               All South Africa North                            │
│                                                                 │
│  ┌───────────────────────────────────────────────────────────┐  │
│  │   App Service B1 Linux  (Express server)                  │  │
│  │                                                           │  │
│  │   ┌───────────────┐  ┌───────────────────────────────────┐│  │
│  │   │  EasyAuth     │  │  Express Web Host                 ││  │
│  │   │  (Entra ID)   │  │                                   ││  │
│  │   │  Dedicated app│  │  /healthz → direct 200            ││  │
│  │   │  registration │  │  /api/*   → proxy to Functions    ││  │
│  │   │  Claims-array │  │  /*       → serve dist/index.html ││  │
│  │   │  format       │  │  dist/assets/* → immutable cache  ││  │
│  │   └───────────────┘  └────────┬──────────────────────────┘│  │
│  └───────────────────────────────│───────────────────────────┘  │
│                                  │                              │
│                                  │  x-internal-proxy-secret     │
│                                  │  x-ms-client-principal       │
│                                  ▼                              │
│  ┌───────────────────────────────────────────────────────────┐  │
│  │  Standalone Functions App  (Consumption plan)             │  │
│  │                                                           │  │
│  │  AUTH_MODE=appservice                                     │  │
│  │  Validates proxy secret → parses claims-array             │  │
│  │  IP restrictions: App Service outbound IPs only           │  │
│  │                                                           │  │
│  │  /api/me  /api/data  /api/ai/chat  /api/etl/upload        │  │
│  │  ... 40+ endpoints (same business logic, new auth mode)   │  │
│  └─────────────┬─────────────────────────────────────────────┘  │
│                │                                                │
│                ▼  (same region, ~1ms)                           │
│  ┌────────────────┐  ┌────────────┐  ┌───────────────┐          │
│  │  Cosmos DB     │  │ETL Extract │  │  Blob Storage │          │
│  │  (same region!)│  │(unchanged) │  │  (unchanged)  │          │
│  └────────────────┘  │ETL Sync    │  └───────────────┘          │
│                      │(unchanged) │                             │
│                      └────────────┘                             │
└─────────────────────────────────────────────────────────────────┘

Improvements:
  ✓ All traffic stays in South Africa — no cross-region hops
  ✓ Express serves SPA + proxies to Functions in same region
  ✓ Dedicated EasyAuth with claims-array auth (no SWA lock-in)
  ✓ Functions independently scalable and monitorable
  ✓ IP-restricted: Functions only accept traffic from App Service
  ✓ Shared-secret trust boundary on every proxied request
  ✓ SWA kept as parked standby for emergency failover
  ✓ Cost: +$13/month for the App Service plan

The Auth Migration

This deserves its own diagram because it was the hardest part of the migration. SWA and App Service EasyAuth present identity differently. The backend had to understand both.

  SWA Auth (before):                    App Service Auth (after):
  ──────────────────                    ────────────────────────
  x-ms-client-principal                 x-ms-client-principal
  │                                     │
  ▼                                     ▼
  Base64 → JSON                         Base64 → JSON
  {                                     {
    userId: "abc",                        claims: [
    userRoles: ["admin"],                   { typ: "oid", val: "abc" },
    identityProvider: "aad"                 { typ: "email", val: "..." },
  }                                         { typ: "roles", val: "admin" }
                                          ]
  Top-level fields                      }
  (SWA-specific)
                                        Claims array
                                        (standard Entra format)

  AUTH_MODE=swa                         AUTH_MODE=appservice
  + no proxy secret needed              + x-internal-proxy-secret required
  + SWA manages the hop                 + timing-safe secret validation
                                        + claims-array parsing
                                        + identity canonicalization

Act 1: The Runbook

Why This Migration Needed a Runbook

This wasn't a "lift and shift." Moving from SWA to App Service touched:

  • 4 Azure resources to provision (App Service plan, web app, Functions app, storage account)
  • 56 app settings to migrate from SWA to the standalone Functions app
  • 12 user identities to canonicalize from SWA format to Entra format
  • A new auth mode (App Service EasyAuth with claims-array parsing)
  • A new web host (Express server replacing SWA's built-in hosting)
  • 2 CI/CD pipelines running in parallel during validation
  • An ETL pipeline that needed seamless ownership transfer between workflows
  • A parked standby mode for the old SWA (not decommission — failover readiness)

The runbook grew to 21 sections with 2,300+ lines. Every Azure CLI command. Every app setting category. Every auth claim extraction rule. Every verification checkpoint.

Seven Review Sessions

Before Codex touched anything, Claude reviewed the runbook across seven dedicated sessions between March 6-26, 2026:

SessionDateFocus
1Mar 6Initial architecture validation and scope framing
2Mar 25 (18:55)Plan validity: are the phases correctly sequenced?
3Mar 25 (19:34)Autonomous execution review: can an AI agent run this without portal clicks?
4Mar 25 (20:03)Architecture and design review: is the auth migration sound?
5Mar 25 (20:36)Implementation plan: critical path validation
6Mar 25 (20:59)Expert engineer review: what would a senior engineer push back on?
7Mar 25 (21:24)Deep review: identity migration, rollback, and edge cases

Key issues caught during review:

  • Identity continuity gap: The initial runbook assumed user IDs would carry over. Claude caught that SWA uses platform-managed service principals while App Service EasyAuth uses Entra object IDs — a completely different identity format. This led to adding the userIdentity.js canonicalization layer and the one-time backfill script.
  • Auth lightweight path: The verifyTokenLightweight function used by AI chat was SWA-only. Without an App Service equivalent, AI chat would break silently after migration.
  • ETL upload streaming: If Express body-parsing middleware was added before the /api proxy, multipart ETL uploads would break. The runbook was updated to explicitly forbid express.json() ahead of the proxy mount.
  • Rollback strategy: The original plan assumed SWA decommission. I later changed my mind and pushed for a parked-standby model instead — keep SWA deployable as emergency failover, not delete it.

Act 2: The Execution

Codex received the runbook and started working on March 29, 2026 at 14:56 SAST.

Day 1: Infrastructure and Code (March 29)

TimeEvent
14:56Inventory capture + code implementation (auth, identity, server, tests)
15:22Azure provisioning: App Service plan, web app, Functions app, storage account
15:41Identity backfill dry-run: 12 users scanned, 12 canonical migrations found
15:42Identity backfill applied: 12 users migrated, zero conflicts
16:43Web deploy (first attempt — Windows zip failed, rebuilt with POSIX paths)
17:00API deploy: standalone Functions packaging fixed, proxy verified
17:06Auth blocker: Entra app registration failed (insufficient tenant privileges)
18:05Full verification: 56/56 config parity, health green, smoke blocked only by auth
19:18Workflow + deploy path hardening committed
22:10API deploy recovery: Functions-action produced 503; switched to source-only Kudu
23:33Kudu false-negative analysis: rsync symlink errors masked a healthy deploy
23:56Both GitHub Actions workflows green. SA web + API deployed successfully.

Day 2: Auth, Validation, and Cutover (March 30)

TimeEvent
14:32Entra auth unblocked: dedicated app registration created with new privileges
14:40EasyAuth enabled: login redirect verified working
15:01Identity re-audit: 9 canonical, 2 clean migrations, 1 overlap detected
15:15Overlap identity fix + verification hardening deployed
15:48Auth flow correction: enableIdTokenIssuance was false, fixed live
16:05Workflow smoke alignment: accept EasyAuth-protected probe responses
17:18ETL admin regression: EtlPipelineView used wrong role authority, fixed
18:24Documentation strategy rewrite: park SWA, don't decommission
21:03ETL ownership switched to SA workflow
21:22Final cutover: SWA parked, SA primary, both workflows green
21:34Failover drill fix: workflow_dispatch jobs were gated to push-only
21:49Failover drill complete: SWA restored, re-parked, verified end-to-end

Act 3: The Battles

Autonomous doesn't mean smooth. Codex hit real obstacles and worked through them.

Battle 1: The Windows Zip

The first web deploy failed because the zip archive built on Windows contained backslash paths. Azure's OneDeploy rejected them. Codex rebuilt the package with POSIX-style paths and redeployed successfully.

Battle 2: The Functions 503

The standard Azure/functions-action@v1 with pre-built node_modules produced a deployed Functions app that returned 503. Codex diagnosed it, switched to source-only Kudu zipdeploy with SCM_DO_BUILD_DURING_DEPLOYMENT=true (matching the pattern already proven by the ETL sync app), and restored the API to 200.

Battle 3: The Kudu False Negative

After fixing the deploy shape, Kudu still reported "failed" — because rsync couldn't create node_modules/.bin/* symlinks. But the app was actually healthy. Codex analyzed the log pattern, hardened the Kudu helper script to recognize this specific false negative, and added a health-gated fallback.

Battle 4: The Tenant Privilege Blocker

Creating the Entra app registration required Application Administrator privileges that Codex didn't have on Day 1. This blocked EasyAuth completely. I resolved the privilege overnight, and Codex resumed on Day 2.

Battle 5: The ID Token Gap

After enabling EasyAuth, browser logins failed silently. The Entra app registration had enableIdTokenIssuance=false, but App Service EasyAuth requests response_type=code id_token. Codex found this, set the flag to true via CLI, and updated both the provisioning and verification scripts to treat it as required state.

Battle 6: The ETL Role Regression

The ETL admin page broke for the admin user on the new stack. Root cause: EtlPipelineView preferred raw tokenRoles (which under App Service auth is just ["authenticated"]) over the database-backed profileRoles. Codex fixed the precedence and added a regression test.


Act 4: The Numbers

MetricValue
Total execution time~28 hours across 2 days
Files created14
Files modified18
Tests passing537 across 149 test files
User identities migrated12
App settings migrated56 (verified parity)
Azure resources provisioned4 (plan, web app, Functions app, storage)
GitHub Actions workflows2 running in parallel, both green
Execution ledger entries30+ timestamped operations
Portal clicks0
Incremental monthly cost+$13 (one B1 Linux plan)

What Got Deployed

  • Express web host serving the React SPA with immutable asset caching
  • API proxy with shared-secret trust boundary and 180s timeout
  • Standalone Functions app with AUTH_MODE=appservice and IP restrictions
  • Dedicated Entra app registration with EasyAuth
  • App Service auth parser with claims-array extraction and timing-safe secret validation
  • Identity canonicalization layer with SWA-to-Entra migration
  • Kudu zipdeploy helper with false-negative resilience
  • Curated deploy artifact with dependency pruning
  • ETL workflow parity with ownership switch variable
  • SWA parked standby with verified failover drill

What I Learned

The Execution Ledger Pattern

The most valuable artifact wasn't the code — it was the execution ledger. Every action Codex took was recorded with timestamp, phase, command, sanitized result, and next action. This append-only log became the working memory across sessions and the audit trail for the entire migration.

When Codex hit the tenant privilege blocker on Day 1 and had to resume on Day 2, the ledger told it exactly where to pick up. When the Kudu deploy shape needed three iterations, the ledger captured each failure and its resolution.

If you're planning autonomous multi-session work, build the ledger into the runbook from the start.

CLI-First Changes Everything

The runbook's execution rule — "no Azure Portal or GitHub UI dependency; all setup must be executable by az, gh, PowerShell, or GitHub Actions" — was the single most important constraint. It made the entire migration automatable.

Every resource provisioned by az appservice plan create. Every secret set by gh secret set. Every EasyAuth configuration by az webapp auth update. Every verification by scripted probes. Zero portal clicks meant zero human bottlenecks.

Park, Don't Decommission

My push toward a parked-standby model instead of immediate SWA decommission was the right call. On Day 2, after cutover, Codex ran a full failover drill: unparked SWA, verified the full app was serving, then re-parked it. The whole cycle took 15 minutes and proved the rollback path works.

For any production migration: keep the old thing alive in standby until you're confident you'll never need it. The cost of maintaining a parked SWA ($0) is much less than the cost of recreating one in an emergency.

Auth Migrations Are Never Simple

We hit five distinct auth-related issues across two days: tenant privileges, ID token issuance flags, claims-array format differences, role authority precedence, and identity canonicalization. Any one of them could have broken the migration silently.

The runbook's detailed auth specification — with pseudocode for claim extraction, validation ordering, and normalized return shapes — was essential. Without it, the agent would have guessed at the auth format and produced subtly wrong behavior.

The Human's Role

I didn't write the Express server, the auth parser, the identity backfill script, the deploy workflows, or the provisioning scripts. But I:

  • Designed the target architecture
  • Wrote a 2,300-line runbook that left nothing ambiguous
  • Reviewed it across 7 sessions with an AI planning partner
  • Resolved the tenant privilege blocker that no CLI command could fix
  • Made the cutover decision based on the verification evidence after confirming with pilot users
  • Performed manual browser validation that proved the stack worked end-to-end

The pattern is the same as before: the human's job is to write specifications precise enough that code writes itself. The better the runbook, the more the agent can do autonomously.


Try This Yourself

Compared to the AI service migration (which was a code extraction and new service build), this SWA migration was a different kind of challenge: less code, more infrastructure, more auth complexity, more operational choreography.

If you're planning a similar hosting migration:

  1. Audit your auth surface before you start. SWA, App Service, and B2C/Entra all present identity differently. Map the claim shapes explicitly.
  2. Build the execution ledger into the plan. Autonomous agents that work across sessions need persistent working memory.
  3. Require CLI-only execution. If the plan needs portal clicks, the agent can't run it.
  4. Run both stacks in parallel. Shared data (same Cosmos, same Blob) means zero data migration. Two active frontends during validation costs almost nothing.
  5. Park, don't delete. Your rollback is only useful if it still exists.
  6. Test the real user path manually. Health probes pass, workflows are green, config parity is 56/56 — and then a human opens a browser and auth fails because of a flag nobody thought to check.

The runbook used in this project is shared in the appendix below.

Appendix: The Complete Migration Runbook (Redacted)

Below is the full runbook that guided this migration, exactly as Codex executed it. Sensitive identifiers — Azure resource names, GitHub references, email addresses, Entra IDs, connection strings, and deployment credentials — have been replaced with <placeholder> tokens. The architecture decisions, execution patterns, CLI commands, and verification checklists are preserved verbatim.

This is the document that Claude reviewed across seven sessions and Codex executed autonomously over two days. Scroll through to see the level of detail that makes autonomous agent execution possible.

Scroll inside the frame to read the complete runbook. The document contains 21 sections covering architecture, auth design, CLI automation, deployment workflows, and verification checklists.


Muhammad Khan is a GM moonlighting as software engineer in his spare time, learning about AI-augmented development workflows, cloud architecture, and autonomous agent orchestration.