The first is irreversible actions. Page 47 describes agents "sending communications, or completing transactions with limited human oversight". On page 48, the filing warns of consequences "that may be difficult or impossible to reverse, such as committing code changes, deleting or modifying data, or completing financial transactions—before effective intervention is possible."

The design response is to classify every tool call by whether it can be undone, and to enforce that at the tool layer, not in the prompt. A prompt that says "ask before sending" is a request. A send tool that only accepts an approval ID is a control. One workable scheme has three classes. A read call fetches something. A prepare call creates a draft, a staged change or a proposed entry, all of which can be thrown away. A commit call sends, pays, submits or overwrites. Reads and prepares can run on their own. Commits need a signed approval that references the exact draft.

tools:
  fetch_client_documents: { class: read }
  draft_client_email:      { class: prepare }
  send_client_email:
    class: commit
    requires: approval_id      # must match the draft hash being sent
    approver_role: staff_reviewer

The second failure mode is error propagation. Page 48: "In an agentic workflow, an error or unintended action at any individual step may propagate through subsequent steps, compounding the severity of harm before detection or correction." Long chains are where this bites. If an early step misreads a document, every later step builds on the bad read.

Two things help. Put cheap validation between steps, such as schema checks, totals that have to tie and required fields that can't be blank, so a bad intermediate result stops the run instead of feeding the next step. And give the reviewer the intermediate artifacts, not only the final draft. A reviewer who can see which source document drove which line can spot the bad read where it happened instead of puzzling over the final draft.

The third is reviewer overload, and the filing is blunt about it. On page 48, agent activity "may also exceed the capacity of human reviewers to meaningfully understand and evaluate whether each action is appropriate", and "Human reviewers may experience alert fatigue, may lack sufficient context to evaluate agent decisions, or may not review agent actions in time to prevent harm". Page 52 adds that oversight "may prove insufficient in practice if the volume, speed, or complexity of autonomous agent actions exceeds the capacity of human reviewers to meaningfully evaluate and intervene".

Much of this is a product design problem. If your approval queue is a flat list with an approve button on every row, you've built the alert-fatigue machine the filing describes. Better queues sort by consequence: a payment above a threshold or a message with a flagged conflict goes to the top with its sources inline, and routine reminders get grouped so one decision covers a reviewed batch. Track time-to-approve and edit rate per category. An approval that's always near-instant is worth a look.

The fourth is intent drift. The filing says on page 48 that "agentic systems may pursue stated objectives in ways that are technically consistent with their instructions but contrary to the user’s actual intent." You can't fully test for that, but you can make it visible. Have the agent state, in one plain sentence, what it thinks the task is and what it's about to commit, and show that sentence on the approval screen. Showing the goal gives the reviewer a chance to catch a misunderstanding before the action runs.

The fifth is the record. Page 49: "maintaining comprehensive records of the actions, decisions, and reasoning of our agentic systems across complex, multi-step workflows is technically challenging", and "any inability to provide adequate audit trails could impair our ability to investigate incidents, respond to regulatory inquiries, or defend against liability claims." If the vendor finds this hard across its own systems, your application should keep its own record rather than rely on provider logs. At minimum that means the inputs and source references, each tool call with its class, the draft hash, who approved it and when, any edits they made, and what was actually sent.

{
  "run_id": "<run_id>",
  "tool": "send_client_email",
  "class": "commit",
  "draft_sha256": "<sha256 of draft>",
  "sources": ["<doc_id>", "<doc_id>"],
  "approved_by": "<reviewer_id>",
  "approved_at": "<ISO 8601 timestamp>",
  "edits": "<count>",
  "sent": true
}

The gate lets preparation keep going while the consequential action waits for review. Reads and prepares run without waiting, and the person's time goes to the step that can't be quietly undone.

Precision starts from the same line between preparing and sending. AI agents for CPA and accounting firms and other professional service firms. Agents prepare the work. Sends wait for your team's approval by default. The product side of the approval step is described here: https://getprecision.ai

The filing is on EDGAR: https://www.sec.gov/Archives/edgar/data/2130357/000162828026067168/anthropicpbc-sx1.htm. Quoted passages are in Risk Factors, pages 47 to 49 and page 52.