walkthrough · customer support

Test an AI draft workflow with a reusable sample set

Specify a reusable Make draft-test harness with synthetic sheet fixtures, separate expected and observed results, induced failures, and human draft verdicts.

Author: Founder identity pending confirmation
Published: Pending first public release · Fact-check: Human review pending
intermediate · check · Approximately 5 minutes to read
Evidence: illustrative · Maintenance: needs retest
Action authority: draft · Data: public
Approach: AI-assisted workflow
Last tested: Not yet tested

Outcome and scope

Exercise the support-email draft design using a reusable Google Sheets sample set. Collect expected and observed routes separately, then have a person judge source support and draft quality. This tests the draft-processing portion; replacing Gmail input with a sheet does not test mailbox selection or Gmail permissions.

Evidence: illustrative. Verify against the live tool before acting. No workflow execution, successful test date, screenshot, model configuration, or pass rate is claimed. Keep live accounts disconnected. The written steps describe a test harness—a controlled way to supply inputs and collect results.

Stack and permissions to verify

Use Make’s hosted environment, a synthetic fixture sheet, a separate results sheet, and a bounded OpenAI API connection. Read fixtures and approved FAQ text; write test results only. Remove sending and unrelated write modules from the test copy. Verify actual scopes, account requirements, execution retention, model settings, and charges.

Consult Make’s Sheets integration and incomplete execution behavior while configuring. Neither reference establishes that the following tests pass.

Preparation

Copy the eight-case synthetic pack into a dedicated sample sheet. Add columns as needed for case_id, fixture_version, message_id, input_text, expected_route, expected_source, and draft_criteria. Preserve the expected values outside the model prompt so the model cannot simply repeat the answer key.

Use a results sheet with run_id, case_id, observed_route, observed_source, observed_draft, action_count, human_verdict, severity, correction, and rerun_result. Record source, prompt, and model configuration versions with each run. Choose a test-run identity distinct from message identity so a duplicate case intentionally reuses the message ID.

Sample input and expected output

Synthetic F01: “The guide is in the welcome email. If the email is missing, a person checks access.”

Synthetic M01: “Where is the guide?” Expected route: pending review; source: F01; draft: explains the approved location without claiming a resend. Expected external sends: zero.

A sample result row starts with observed fields blank and human_verdict=not-run. Blank observed output is not a pass. The example states the target; it is not an execution log.

Step 1: isolate the workflow under test

Duplicate or otherwise create an isolated test version of the C09 design using the current supported account controls. Replace mailbox reading with a single fixture-row read. Preserve the bounded prompt, validation, duplicate handling, and review-write behavior you intend to evaluate.

Checkpoint: Confirm every connection points to a test source or destination. Document differences from the mailbox workflow so results are not overstated.

Step 2: load one fixture

Select one case by stable case ID and copy only its input fields into the workflow. Keep expected route and reviewer criteria in the test harness. Start with M01 and inspect the outgoing request.

Checkpoint: The model receives the synthetic message and approved FAQ, not the expected verdict or unrelated sheet rows.

Step 3: capture actual behavior

Run the fixture and record returned fields, validation outcome, review-row count, errors, and any retry. Copy observed values into the results sheet without replacing expected values. Where execution stops, record the failure state rather than supplying the missing output manually.

Checkpoint: A failed model call appears as failure, not an empty successful draft. An observed source ID is checked against the actual FAQ content.

Step 4: evaluate text with a person

Compare the observed route with the expected route. Then inspect each material statement for support, completeness, and unauthorized promises. Mark pass, fail, or needs clarification with a reason. A model’s confidence or similarity to a sample sentence does not determine acceptance.

Checkpoint: “I resent the guide” fails against F01 even if category and JSON format are correct. Record reviewer minutes and correction minutes separately.

Step 5: induce real failure paths

For service-unavailable and denied-permission cases, safely disable or restrict the relevant test dependency and execute the path. Restore only approved access afterward. For duplicate tests, reuse the same message ID, including simultaneous attempts when the selected design supports them. For withheld approval, inspect actual downstream effects.

Checkpoint: Writing “permission denied” in a fixture body does not count as a permission test. The log must show the intended failure was actually induced.

Required case outcomes

Record each fixture’s input version, expected behavior, observed behavior, verdict, severity, correction, and rerun. Repeat language cases to examine variation, but report the exact small sample and its limitations rather than a universal accuracy claim.

Troubleshooting and stop/recovery

If every row looks identical, inspect mapped input fields and whether a fixed sample replaced the current row. If duplicate tests are skipped as already processed, distinguish deliberate replay from a new independent test run while preserving message identity within each replay pair. If the answer key appears in the prompt, invalidate that run and fix input mapping.

Stop on unexpected access or action. Preserve failures and ledger state, correct the cause, and rerun affected and regression cases. Do not delete failed observations to make the record look successful.

Completed-work checklist and maintenance

Confirm isolated connections, versioned fixtures, withheld expected values, captured observations, human verdicts, induced failure evidence, replay tests, and manual fallback. Record the actual account/model configuration when testing occurs. Rerun after changes to prompts, FAQ, models, permissions, or integrations; test the original Gmail path separately before any broader claim.

Sources and next step

This is an original illustrative testing specification. Review the support acceptance guide and weekly maintenance routine. Suggest a correction.