Evaluate an agent by what happens between a request and a completed case, not by model access alone. A model can interpret an instruction and propose a next step. An agent harness is the operating layer that assembles context, tracks workflow state, mediates permitted tool use, places required approval before consequential action, handles exceptions and preserves a record an operator can understand. Model suitability still matters. The decision is whether the model sits inside controls that make its work bounded, recoverable and inspectable.
The operating mechanism
Consider one illustrative service case: a request to correct an address in an account record. Completion means the authorised change is confirmed in the source system and the case closes with a usable record. The model can interpret the request and draft the next step. The harness determines which policy and case data are brought forward, where the case stands, which system action is permitted, whether approval is required and how an uncertain result is resolved.
Its responsibilities are functional, even when products package them differently:
- Context and state provide relevant information for the current step and track what has happened across steps and interruptions.
- Tools let the workflow read or change defined systems.
- Permissions and limits constrain identities, actions, attempts and spend.
- Approvals place human authority before a consequential action and attach the decision to the case.
- Recovery defines what to do when information is missing, a tool fails or an outcome is uncertain.
- The record connects the instruction, source context, actions, approvals, interventions and final outcome.
Every attempt consumes compute and operating attention, regardless of how model capacity is bought. Poorly selected context can cause rework; lost state can restart completed steps. If an update times out, a blind retry can create a duplicate. Recovery should first establish whether the original action occurred, then resume, route an exception or ask an operator. A coherent record makes the outcome easier to reconstruct and the workflow easier to maintain. These controls can influence completion, human effort, operating cost and exposure. They do not replace sound process design, reliable data, suitable models or accountable people.
Keep the value claims separate. Realised cash requires an observed, attributable net movement in money paid or received. Capacity is measured time released, whether redeployed, absorbed or unused; multiplying hours by salaries does not make it cash. Structural value is a durable improvement in control, resilience or maintainability. Modeled upside remains prospective until recognised and attributed. None should be added into a blended total.
The decision
Evaluate candidate platforms against the requirements of the address-correction case before comparing the breadth of model access. Specify the approved sources, freshness rules, workflow checkpoints, tool identity, permitted action, limits, approval point, exception route and record needed at closure. Ask what an operator can inspect or reset, and what extra engineering and operating work would fill any gap. Missing controls do not disappear during procurement; they move into implementation and ongoing ownership.
Portability belongs in the same assessment, but a connector list is not proof. Replacing a model or tool may require interface changes, prompt or policy updates, data mapping, regression tests and renewed approval. A credible comparison identifies that work and tests the transition path rather than treating every switch as a configuration change.
What would count as proof?
Start with a documented comparison of the controls above, plus portability and what an operator can observe. Use the same workflow, comparable case mix, current-process baseline, completion definition and service and control thresholds for each candidate. Name the process owner. Run normal cases as well as missing information, a rejected action, a deliberately forced timeout and an interruption. Trace each case from intake to an authorised source-system update and closure. Verify source freshness, permission enforcement, approval timing, recovery and operator reconstruction. Record completion, exceptions, human touches, elapsed time, direct platform cost and the work required to build and operate the controls. Test a model or tool replacement and record the changes and revalidation it requires. Attribute any outcome against the baseline without double counting.
What remains unclaimed?
A passed test supports a narrow conclusion: the operating layer can handle the specified case under the tested conditions. It does not establish production reliability across other workflows or case mixes, correct judgement in every instance, automatic adoption, universal safety or financial return. A readable activity record is not automatically audit evidence. Portability in one transition does not prove portability everywhere. Realised cash, released capacity, structural value and modeled upside each require their own evidence after operation begins. The harness enables control; it does not guarantee the outcome.