Future Frontiers · Issue 17

Five questions. Pick the best answer for each. Scoring at the bottom.

1. Your team wants to deploy an agent to handle inbound customer-service tickets. The most important early investment is:

a) The largest foundation model your budget allows

b) A clearly scoped permissions model — what the agent can read, write, and send

c) A bigger context window so the agent can see more ticket history

d) A custom fine-tune on your historical tickets

2. A vendor pitches their product as “fully agentic.” The strongest screening question is:

a) “What foundation model do you use?”

b) “If we remove the next step from the prompt, does the system still complete the task?”

c) “Is your context window at least 200K tokens?”

d) “Do you support fine-tuning?”

3. Your agent has 95% per-step accuracy across an eight-step process. The end-to-end success rate is approximately:

a) 95%

b) 88%

c) 66%

d) 40%

4. Which is the strongest predictor that an agent deployment will fail in production?

a) The foundation model is from a competitor

b) The team can’t replay the agent’s plan, tools called, and inputs/outputs

c) The agent uses a smaller context window than the latest frontier model

d) The cost per task is higher than the human baseline

5. Six months into deployment, your agent’s average cost-per-task has tripled. The most likely cause is:

a) The vendor raised prices

b) The team is sending more tokens per call than they realize — longer prompts, more tools, deeper retrieval

c) The model got worse

d) Volume increased

---

Scoring

• 5 correct: Production-ready. You’re thinking about the right problems. Pick your first hard use case.
• 3–4 correct: Pilot-ready. The fundamentals are there; close the specific gaps before scaling.
• 1–2 correct: Foundation-building. Spend a quarter on observability, permissions, and one well-scoped pilot before greenlighting more.
• 0 correct: Hire someone, or hire a partner. The fastest path forward is bringing in someone who has done this once before.

Auto-reply from: copilot@yourcompany.com

Hi —

Thanks for your message. I’m currently out of context.

I’ve received your prompt, but my last 47,000 tokens included an entire SQL schema, two HR policy PDFs, and the full text of a deal binder that wasn’t really relevant to your question. I can technically still respond. The relevant parts of your request are now in the middle of my attention span — which, as you may know, is the part I’m worst at remembering.

For the fastest response, please:

• Re-send your question in a fresh thread, without the schema and the PDFs. I will get to it.
• Be specific about what you actually want. “Help me with the report” gives me approximately the same context as “help me with life.” Both are technically answerable.
• Don’t ask me to remember anything from before today. I will say I remember. I will be lying. It will be convincing.

For urgent items, please contact a human, ideally one who has been at the company for more than six months and can still reliably tell you which printer to use.

I expect to be back in context shortly. Or in about 800 milliseconds, whichever comes first.

Best,

Your Copilot

This auto-reply was generated by your Copilot. If the tone seems off, that’s because my system prompt was last updated by someone who has since left the company.