FF/Newsletters/Issue 19

Issue 19 · August 1, 2026

Future Frontiers · Issue 19

Why claims keeps disappointing the people who bet on it, the small mechanism that turned chatbots into agents, and the arithmetic that quietly undoes a four-step loop.

Etcetera

Tidbits of levity, fun, and relief from the serious work.

04 · Picture This
Production-grade.

The long road from demo to "production-grade."

05 · Brain Teaser

The Cascade

A claims-triage agent runs a four-step loop on every incoming claim: extract loss details (97% accuracy), classify loss type (96%), verify coverage (94%), assign adjuster (95%). Each step has been individually validated and meets the 95% accuracy bar set by ops.

The team ships it. After six weeks in production, the audit team flags that 17% of fully-routed claims have at least one error somewhere in the chain.

The ops team double-checks the per-step numbers. They’re correct: each step is at or above 94%.

Why is the end-to-end error rate 17% when no individual step is below 6%?

Answer at the foot of the issue ↓

06 · Just in Jest

An Agent's First Day, in Notes to Self

Claims Triage Agent · Deployed 9:04 a.m. · Notes to self

9:04 — Read the process documentation. Thirty steps. Elegant. Confident this will take the morning.

9:11 — Step 4 says verify coverage. Step 5 says proceed. Between them, in the actual work, there appear to be nineteen things. None of them are in the document.

9:40 — Located the team’s shared drive. Found a folder named FINAL. Inside it: FINAL_v2, FINAL_v2_USE_THIS, and FINAL_old_DO_NOT_USE_v3. Escalating.

10:15 — Asked what to do with a claim where the policy number is correct and the name on the form is a nickname. Was told, “oh, just ask Deb.” Deb is not in my tool registry. Adding Deb to my tool registry.

11:02 — Deb answered in four seconds. Deb has been doing this for nineteen years. I have been doing this for two hours. We are going to be friends.

11:48 — Learned that “the system is slow today” and “the system is down” are entirely different conditions, and that the difference is worth about forty minutes.

2:15 — Cleared sixty-one claims. Flagged four. Three were genuinely ambiguous. The fourth involved a duck. Not a metaphor. An actual duck.

3:40 — Discovered a sticky note on a monitor that describes an exception path more accurately than anything in the knowledge base. It reads, in full: “if pink form, DON’T.” Have added this to my instructions verbatim. It is now my single best-performing rule.

4:50 — End-of-day tally. Documentation: thirty steps. Observed: two hundred and eleven. Deb says that’s a slow day.

5:00 — Tomorrow: ask Deb about the pink form.

Filed by the Claims Triage Agent. Reviewed by Deb, who added: “It’s doing fine. Somebody tell it about the pink form.”

Brain teaser · answer

The Cascade

Errors compound. The probability that all four steps run without error is 0.97 × 0.96 × 0.94 × 0.95 ≈ 0.831 — meaning ~17% of claims have at least one error somewhere in the chain. The audit and the per-step numbers are both right.

This is the Cascade. Per-step accuracy tells you almost nothing about a multi-step agent; what counts is end-to-end task success, measured at the boundary the customer or the auditor actually sees. A 95%-per-step agent is 81% reliable over four steps, 60% over ten, and 36% over twenty. So either shorten the chain, or pick problems where you can catch the error and recover.