A useful pilot should leave you with a decision: continue, change the approach or stop. It should also leave the business able to work normally tomorrow.
Start with one modest task. In the hypothetical example below, a VA prepares internal next-action summaries from approved service-request notes. An AI tool drafts the summary; a person checks it before anyone relies on it. The original request remains the authoritative record.
This is an original 30-day planning framework, informed by guidance checked on 25 September 2026. The dates are review points. Progress depends on evidence and readiness, so a stage may take longer or end the pilot.
Choose one task and screen the proposed use
Define the output: a short summary containing the request reference, established facts, outstanding question and recorded next owner. Unknown information stays visibly unknown. The pilot grants no authority to promise dates, approve refunds or resolve technical questions.
Use the National AI Centre's screening questions before committing resources. They identify uses that may need additional controls; the questionnaire has no automatic pass score and does not replace a full risk assessment. NAIC: AI screening questions.
Name a pilot owner, trained operator, reviewer and backup. One person may hold multiple roles, provided review time and authority are real. Use the hybrid team responsibility chart to clarify handovers.
Write a one-page brief covering permitted inputs, tools, exclusions, review budget and who can authorise each stage. Consider whether a clearer template would solve the problem before adding AI.
Days 1–7: establish the baseline and acceptance rules
Observe the existing manual task across routine and awkward requests. Record preparation, checking, corrections and accepted summaries. Also record request complexity and delays waiting for decisions. Keep active work separate from elapsed waiting.
Agree what the experiment will test. For this fictional workflow, a 20-case test set could use these proposed criteria:
| Check | Example pilot acceptance condition |
|---|---|
| Factual fidelity | No invented dates, commitments or owners in any test draft |
| Missing information | Every deliberately incomplete case identifies the gap |
| Usability | At least 18 of 20 summaries need no substantive rewrite |
| Human control | Every output remains a draft until reviewed; no automatic sending or status changes |
| Operating effort | Combined preparation, review and correction time meets the budget agreed before testing |
These are illustrative pilot thresholds, not official standards or proof of reliability. Record both raw-draft defects and final accepted quality. Fixing an error before release protects the workflow, but the error still belongs in the trial results.
Prepare the stop rules and manual fallback now, before testing begins.
Days 8–14: test beside the existing workflow
Begin with fictional records that reflect common cases, missing details and conflicting notes. In shadow testing, the established manual process continues to produce the working summaries. AI drafts sit separately for comparison and cannot direct real work.
Before any real information enters the tool, review its suitability, access and data handling. OAIC guidance recommends considering these matters and, as best practice, keeping personal information out of publicly available generative AI tools. OAIC: commercial AI guidance.
Save the approved instruction, tool configuration, input reference, draft, reviewer finding and correction. Change one identifiable part of the approach at a time. Keep some unseen cases for retesting so improvements are not assessed only against examples used to refine the prompt.
At the review point, examine failures and reviewer effort. A fluent summary that assigns the wrong owner fails the agreed test. Remain in shadow testing until the owner has evidence supporting a limited live trial.
Days 15–21: run a small supervised queue
If the previous gate is met, authorise a capped live workload that the reviewer can check completely. Document the permitted records and access before starting. The cap should reflect available review capacity, not the number of drafts the software can produce.
For every summary, the VA selects approved source material, checks the draft and routes unresolved points. The assigned reviewer approves its use. Keep technical, financial and other specialist decisions with the appropriately authorised person.
At each handover, record accepted outputs, rejected drafts, unresolved cases and any effect on customer work. Continue recording source preparation, checking, coaching, correction and tool-maintenance time. Include the work needed to recover from a failure.
NAIC recommends testing before deployment and monitoring afterwards. It also supports human intervention and alternative pathways when systems fail. NAIC: testing and human control.
Pause safely when a stop condition occurs
For this pilot, pause the affected AI step if information enters an unapproved tool, an output bypasses review or the system repeatedly invents commitments. Also pause intake when the reviewer is unavailable or the queue exceeds the agreed cap.
Record what happened and alert the named owner. Identify affected drafts and whether anyone has already relied on them. Follow the business's incident procedure where applicable; correcting one visible draft may not resolve the underlying problem.
Return new work to the documented manual process. Keep request references intact so tasks are neither lost nor duplicated. Practise this handover with the backup during the trial.
Restart only after the owner records the cause, corrective action and satisfactory retest. Passing a calendar date does not clear a stop condition.
Days 22–30: decide using the full cost of accepted work
Compare similar requests against the same acceptance rules. Measure the whole process, including review, correction and queue delays. Record any case-mix or staffing change that weakens the comparison.
For illustration, reducing total handling from eight to seven minutes across 30 accepted summaries saves 30 minutes at that step. If setup and training took five hours, that small trial has not recovered the initial time investment. Record recurring monitoring and recovery effort separately too. These figures are hypothetical, not an ROI forecast.
Choose to stop, revise and retest, continue at the same scope, or expand one defined element. Record the evidence, remaining uncertainties, owner and next review. Limited test results support a local decision, not a guarantee about other workflows.
Talk to Nexus about a focused VA and AI pilot with a task, reviewer and measurable finish point your team can support.
