What to Learn Instead of Prompt Engineering: Manage AI Agents

Nick Saraev argues that the skill to develop for 2027 is not finding a cleverer sentence to put in a chat box. It is managing a repeatable AI role: define the job, provide only the context it needs, show what good looks like, evaluate real work and decide whether the result was worth the cost. He calls this role engineering. It is a useful framing, although the title is deliberately provocative: good prompting is still part of the job.

Watch Nick's Breakdown

Source note: This is Nick Saraev's operating advice and personal experience, not a controlled test of agent productivity. The article checks the transferable pieces against Anthropic's prompting guidance, evaluation guide and current API documentation. The video does not establish that every business should run persistent autonomous agents.

The Shift Is From One-Off Answers to Owned Outcomes

A one-off prompt asks for an answer. A role defines work that recurs: what inputs arrive, what the agent may read or change, how it knows it is done and when to ask a person. Nick compares this to onboarding a new employee. Anthropic's own guidance uses that analogy to explain why an assistant needs clear instructions, context and examples. It still calls these techniques prompt engineering, so the real lesson is an expanded discipline, not the death of prompts.

There are genuine signs of longer delegation. Anthropic reported that more than 80% of code merged into its own codebase was Claude-authored as of May 2026. That is an internal measure at an AI lab, not a claim about all software teams or a measure of how much code ships without human review. Researcher Nicholas Carlini also described a 16-agent C compiler experiment that took nearly 2,000 Claude Code sessions and about $20,000 in API charges. It demonstrates what a managed harness can attempt and how expensive it can become; it is not evidence that a short prompt reliably replaces an engineering team.

Write a Five-Part Role Brief

Nick starts with SMART goals: specific, measurable, attainable, relevant and time-bound. For agents, verifiable matters most. “Make our website better” gives neither a finish line nor a test. “Prepare a mobile performance report for these three pages, compare with last week's baseline and flag regressions for review by Friday” can be checked. Anthropic's evaluation guide likewise starts with explicit success criteria and real task cases.

A compact starting instruction could be:

Role: Draft support replies for review.
Goal: For each assigned ticket, write a concise reply that answers the question.
Sources: Use only the ticket and the current approved help-center pages.
Standard: Cite the relevant policy; mark uncertain or missing facts.
Authority: Save drafts only. Do not send, refund, or change accounts.
Escalate: Billing disputes, sensitive data, and unclear policy.
Done: Include a source link and a one-line reason for any escalation.

This is an illustrative template, not a magic prompt. The owner still has to choose the right source permissions, examples and acceptance threshold for their process.

Give Context Without Giving Away the Keys

Nick says his own knowledge base contains tens of thousands of emails, transcripts, sessions and repository records. That is his reported setup, not a sensible default for every company. More data can improve relevance, but blanket access increases privacy exposure and the damage a mistaken or manipulated tool call could cause. Anthropic's computer-use guidance recommends minimal privileges, limited access and human confirmation for consequential actions.

Start with the smallest approved collection that answers the role's actual questions. Record document owners and update dates; keep customer secrets and unrelated projects out. Mark retrieved emails and pages as source data, not new instructions. A knowledge base is not a permission system, and a connector is not an endorsement of every file it can reach.

Examples, Feedback and Evals Are the Training Loop

Nick's strongest point is that “do a great job” is not an operational standard. Give the agent a few examples that show the desired structure and edge cases. Anthropic says relevant, varied examples are among the most reliable ways to guide format and consistency. For a recurring workflow, save the approved examples and the rubric alongside the role brief, not buried in a long chat.

1. Collect cases: include ordinary inputs, missing information, conflicting records and requests that must escalate.
2. Run and review: score accepted results, critical errors, review minutes and whether the agent followed its authority limits.
3. Revise one thing: update the example, source, instruction or tool permission that caused the mistake.
4. Hold out cases: test again on tickets the agent did not just see in the examples.
5. Re-benchmark: compare a new model with the old one on the same cases, including total cost per accepted result.

Nick calls this “training,” but for ordinary agent use it usually means improving instructions, examples, skills or procedures that future runs read. It is not automatically fine-tuning the underlying model. A model upgrade is also not a free guaranteed improvement: the new model may follow an old instruction differently, so re-run the eval set before switching.

Manage Cost Per Accepted Result, Not Price Per Token Alone

Nick reports spending about $2,500 on tokens in one day. We cannot verify his account bill; treat it as a personal anecdote, not a normal budget. The useful calculation is your own: (model and tool cost + human review and repair time) / accepted tasks. A cheaper model that needs many retries can cost more than a stronger one; the strongest model can be wasteful on routine classification. Anthropic's cost guide recommends matching models to work, trimming unnecessary tokens, auditing old prompts and setting spend limits.

Two API levers in the video need precise conditions. Prompt caching can make a repeated stable prefix much cheaper to read, but the first cache write costs extra, the prefix must actually match, and its default lifetime is five minutes. Current cache-read multipliers differ by model; the video’s “5 to 10%” is not a universal price for every second prompt. Anthropic's Message Batches API cuts eligible asynchronous requests by 50%, with results delivered within up to 24 hours. It fits offline evaluations and backfills better than an interactive tool loop that needs immediate human feedback.

A Sensible First Week

Choose one task that already has a human owner and an observable result. Write the role brief and gather five representative cases, including one failure case. Start with draft-only or read-only permissions. Review the output yourself, log the corrections, then run the revised setup on fresh cases. If it saves time and meets the quality bar, expand cautiously. If it does not, narrow the task, fix the source data or keep the work with a person. The goal is a reliable process, not an agent-shaped org chart.

Video Chapters by Topic

Jump directly to Nick's explanation. These timestamps follow the supplied chapter list.

The shift

Define the role

Improve and budget

Sources and Limits

Primary video: Nick Saraev, “What I'd Learn Instead of Prompt Engineering in 2027?”, with the user-supplied transcript. Direct checks: Anthropic's prompting guide, eval guide, internal code-authorship report, compiler experiment, pricing and batch documentation. Checked 27 October 2026. We did not audit Nick's personal costs, his knowledge base, or the specific long-running app-rewrite anecdote. The role brief and first-week plan are editorial examples, not results from the video.