The Automation Analyst

From WFM Labs
The analyst reads the written-down process and produces four things — what the agent may touch, what "done" means, the specification, and a test on the operation's own cases — then hands the agent to the overseer, who proves it. Odd cases come back and the specification is revised.

The automation analyst is the role in a workforce management function that converts a documented process into a running AI agent. It takes a process that has been decomposed to the L0–L3 standard, derives from the step table what the agent may touch and what "done" means, writes the agent's specification, tests the agent against realistic scenarios drawn from the operation's own work, and hands it to the agent overseer for the overseen phase. In the terms of the agentic handover gate, the analyst is the person who carries a process from test one to test three; the overseer owns test four, and test five — reversibility and the trigger log — is a property of the record both maintain. The role is defined here because the wiki's existing automation roles sit on either side of the conversion — the automation strategist governs the rule library and the automation orchestrator tunes what already runs — and none of them owns the act of turning a written process into an agent.

Why the role exists

A function that has documented a process and appointed an overseer still has nobody whose job is the conversion between them. In practice the work falls to whoever is nearest: a real-time analyst who knows the platform, a vendor's implementation team, or the process owner. Each produces a different agent from the same document, and none is accountable for whether the agent does what the L2 table says. Early practitioner accounts of service automation found the same gap in robotic process automation programs: the scarce capability was not the automation tooling but the people who could translate a business process into an automatable specification and own its testing, and the recommended model placed that work with business-side process analysts rather than with the technology function.[1] A broader survey of enterprise AI projects reached a compatible conclusion: the projects that succeeded took an incremental approach to well-understood processes rather than starting from the capability of the technology.[2]

The design argument for the role runs the same way from the tooling side. To the extent that agent-building tooling takes over the integration work, what remains is the work no tool does: deciding what the agent is for, deriving its boundaries from the process, and proving on the operation's own cases that it behaves. That is analysis, not engineering, and the role is named accordingly.

The conversion, step by step

Step What the analyst does What it reads What it produces Gate test
1 Confirms the process is decomposed to L2 with L3 instructions for every step the agent will run, and sends it back to the process owner if not The L0 cover sheet, the L1 flow, the L2 table and the L3 register A go or a return, with the missing steps named 1 · documented
2 Derives the permission map: the systems and actions the agent needs, read from the L2 table's Tools Used, Ticketing Code, Notification Required, Approval Required and Approvers columns, and nothing beyond them The L2 table The agent's permission set, for its record 1
3 Reads the definition of done from the L2 table and inherits the objective the process serves — cost, experience, revenue, compliance — from the objective register, then confirms each objective has an instrument on the agentic node. The analyst does not define the objective; the process owner and the overseer own that, and the analyst records whether it can be measured L2 step descriptions; the objective register The measured attributes the gate's second test requires, or a cannot run naming the missing instrument 2 · instrumented
4 Writes the agent specification: the steps, the inputs, the decision points that stay with a person, the exceptions that go to the overseer's queue, and the reversal path The L1 flow and L2 table A specification the overseer and the process owner both sign 1–2
5 Builds or commissions the agent and tests it against realistic scenarios drawn from the operation's own cases, including deliberately difficult and out-of-scope ones, and records reliability under repetition Historical cases; the L3 instructions as the oracle A scenario test record on this work in this domain, not a vendor benchmark 3 · measured here
6 Hands to the overseer with the specification, the permission map, the test record and a proposed error-injection rate for the overseen phase The handover package 4 begins
7 Returns when the overseer's adjudication record shows a gap in the specification, and revises it — the loop that turns exceptions into documentation; each revision is entered in the trigger log so the handover stays reversible The exception record A revised specification and, where needed, a revised L2 table with the process owner 5 · logged (jointly with the overseer)

Step 5 is the test that distinguishes the role from a builder. Agent performance is domain-dependent and degrades under repetition: on a standard tool-agent benchmark the same model performs at roughly half its retail-domain level in an airline-servicing domain, and the probability of succeeding on all of several independent trials of one task falls sharply.[3] An agent passed on a published evaluation rather than on the operation's own scenarios has not been tested.

Delimitation from adjacent roles

Role Owns Does not own Page
Process owner The process as documented: the L1 flow, the L2 table, the definition of done The agent Process Decomposition (L0–L3)
Automation analyst (this page) The conversion: permission map, specification, scenario tests, the handover package, and revision when exceptions reveal a gap The rule library; the running agent's daily tuning; the catch rate
Automation strategist The rule library, change control, and the bridge between operational constraints and platform capability The specification of any one agent Level 3: The Automation Layer
Automation orchestrator Daily tuning of rules and thresholds already in production; threshold sweeps and holdouts Conversion of new processes Level 3: The Automation Layer
Prompt engineer Prompts, knowledge content and configuration governing customer-facing AI behavior Process-derived specifications for ROC agents; the permission map Workforce Planning for AI-Augmented Roles
Agent overseer Custody, the catch-rate program, adjudication of exceptions, the decision to move a process between bands; with the process owner, the objective the agent must move Building or specifying the agent The Agent Overseer
Human–AI orchestration manager Routing logic and governance policy across agents Any one agent's specification WFM Roles

The analyst and the overseer form a pair around each process: the analyst produces the agent and revises it; the overseer proves it and decides its band. The objective the agent must move is defined by the process owner and the overseer, as the overseer page states; the analyst inherits it and reports whether it can be instrumented. Neither role can do the other's job without a conflict of interest — an analyst who published their own catch rate would be marking their own work.

Where the role sits

In a function organized by planning horizon, the analyst sits in the execution layer beside the automation strategist, and is assigned to processes rather than to a horizon; the first assignments are the candidates the gate page names for a first handover — assembling the monthly capacity outlook, publishing schedules, intraday reallocation within guardrails — each of which enters the overseen band on handover and moves to specified only when its catch rate is published. For supplier-delivered work the role may sit with the supplier under the contracted oversight clause, with the buyer's analyst owning the specification standard. The role reports to whoever owns the rule library, because the specification of an agent is a change to it.

Sizing and skills

The role is sized by processes in conversion, not by agents in production: one analyst carries two or three processes through steps 1 to 6 at a time, and a function beginning its first handovers needs one, not a team. The skills are decomposition literacy (reading and, when needed, correcting an L2 table), platform literacy (what the routing, scheduling and workforce platforms expose, and what they cannot represent), test design (building scenario sets that include the cases the process was not written for), and the discipline to return an undocumented process rather than fill the gaps from experience. The last is the reason the role is better filled from the planning function than from a technology team: the analyst must know what the process is for.

First six months

  1. Month one: the first candidate process is taken and its documentation confirmed, and returned if the L2 table is incomplete. This alone usually produces the first finding — that the process everyone knew was not written down.
  2. Months two and three: the permission map, the definition of done and the specification are produced; the specification is agreed with the process owner and the overseer; every objective is confirmed to have an instrument, or a cannot run is recorded.
  3. Month four: the agent is built or commissioned and tested against the operation's own cases; the test record is produced.
  4. Month five: the handover package goes to the overseer and the overseen phase begins; the analyst's work on this process becomes revision on demand.
  5. Month six: the second and third candidates begin, and the first process's exception record is read for specification gaps.

Failure modes

  • Filling gaps from experience. The analyst completes an incomplete L2 table from what they know rather than returning it. The agent then encodes one person's practice as the standard, and the gate's first test has been passed on paper only.
  • Permissions by convenience. The agent is given the analyst's own access rather than the set derived from the tools column. Every documented failure mode of delegated agents lives in that gap (Agent Identity and Custody).
  • Testing on the benchmark. Step 5 passed on a published evaluation rather than on the operation's own cases, when domain alone can halve the result.
  • Analyst as overseer. One person specifies, builds and publishes the catch rate. The ironies of automation apply, and this page adds a second irony to the first: the designer has left the operator with the cases the design could not handle — and here there is no operator.[4]
  • Engineering the role. The role is filled from a technology team by someone who can build but does not know what the process is for; the specification is technically correct and operationally wrong.

Maturity Model Position

The role is created at the transition from Level 3 to Level 4, when the first process is taken through the handover gate; the standardization stage ends with that handover, so the analyst is among the first roles it needs filled. At Level 5 the build is largely tooling and the role's remaining content — specification, scenario design, revision from the exception record — is what the AI–human collaboration designer on that page carries, the role that decomposes processes and assigns human, AI or hybrid ownership to each part.

See Also

References

  1. Lacity, M. C., & Willcocks, L. P. (2016). "A New Approach to Automating Services". MIT Sloan Management Review 58 (1), 41–49.
  2. Davenport, T. H., & Ronanki, R. (2018). "Artificial Intelligence for the Real World". Harvard Business Review 96 (1), 108–116.
  3. Yao, S., Shinn, N., Razavi, P., & Narasimhan, K. (2025). "τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains". International Conference on Learning Representations (ICLR 2025); arXiv:2406.12045.
  4. Bainbridge, L. (1983). "Ironies of Automation". Automatica 19 (6), 775–779. doi:10.1016/0005-1098(83)90046-8.