How should a tester pair with AI?

Pairing is an allocation of judgement

In pair testing, the assistant can suggest questions, cases, data, code, explanations, or the next exploratory action. The tester decides which risk matters, which evidence is authoritative, whether the proposal is permitted, and what observation would establish a defect.

That division is more useful than “AI does the boring work.” A repetitive transformation can be dangerous when it touches production credentials. A creative brainstorming task can be safe when every candidate is reviewed. Delegate according to uncertainty, consequence, and observability—not how impressive the output looks.

flowchart LR
  H[Tester sets mission and risks] --> M[AI proposes cases or actions]
  M --> H2[Tester selects and sharpens]
  H2 --> X[Execute in controlled state]
  X --> E[Observe evidence]
  E --> H3[Tester decides and updates model]
  H3 -->|continue| M

Four productive roles for the assistant

Expand: propose partitions, boundary values, failure modes, personas, state transitions, misuse cases, and questions the tester may have missed.

Transform: turn a user story into a checklist, a schema into candidate data, a manual path into draft automation, or a failure trace into a concise reproduction outline.

Challenge: argue against an assumption, find a counterexample, compare the requirement with the implementation, or generate a plausible faulty implementation that should be caught.

Navigate: search unfamiliar code or documentation, summarize candidate evidence with provenance, and identify where a behaviour is implemented.

All four produce hypotheses. The result becomes testing evidence only after it is grounded, executed, observed, and reviewed.

Keep five responsibilities human-owned

The tester remains accountable for:

1. Mission: why this testing matters and which product risk it addresses.
2. Oracle: what makes an outcome correct, incorrect, or uncertain.
3. Authority: which data, accounts, tools, environments, and effects are permitted.
4. Contradiction: how conflicts between requirements, code, documentation, and observed behaviour are resolved.
5. Acceptance: whether the evidence supports a bug report, release decision, or maintained test.

An agent harness can enforce parts of authority and execution, but it does not decide the organization's values or product risk tolerance. A model can critique its own answer, but that is not independent review.

Choose autonomy by observability

Use direct chat for open questions and early exploration. Use inline completion for small local transformations. Use an agent when the task needs a feedback loop through files, terminals, APIs, or a browser.

Increase autonomy when the environment is disposable, state can be reset, effects are bounded, success has strong deterministic checks, and the trace is inspectable. Reduce it when the task is ambiguous, the oracle is subjective, data is sensitive, actions are difficult to reverse, or a plausible wrong result is expensive.

A useful delegation test is: can we detect a wrong proposal more cheaply and reliably than producing the result ourselves? If review requires reconstructing the entire problem, delegation may create work rather than remove it.

A practical pairing rhythm

Begin with a short charter: target, risks, starting state, constraints, and evidence expected. Ask the assistant for a small candidate set with assumptions visible. Select cases based on risk rather than volume. Execute them against controlled state. Ask the assistant to organize observations, but preserve raw evidence. Challenge the strongest conclusion with a counterexample or mutation before accepting it.

For generated automation, inspect selectors, setup, cleanup, assertions, and failure messages. Run the test against the unchanged product, then prove it fails when the intended behaviour is broken. For exploratory testing, keep the exact starting state, actions, observed result, expected basis, and reproducibility notes.

Use the assistant to explain unfamiliar material, but verify important claims against primary or current first-party sources. Fluent teaching language can conceal a fabricated API or obsolete limitation.

Watch for collaboration failures

Automation bias appears when the tester accepts a polished suggestion without sufficient evidence. Anchoring appears when the first generated list narrows later exploration. Correlated error appears when the same model drafts the implementation, test, and expected answer from the same mistaken interpretation.

Other warning signs are growing review queues, repeated repair prompts, tests nobody can explain, untracked sensitive data in conversations, and sessions that cannot be reproduced after the model or repository changes.

Measure the pair as a workflow. Compare accepted findings, fault detection, human correction time, review cycles, severe misses, elapsed time, attention, and cost with the existing baseline. Research results on AI coding productivity differ sharply by task and setting; local evidence should determine the delegation policy.