# Test 002 Draft - Two Website Proposals to Decision Brief

Status: invalid pilot completed; not published. A corrected v2 specification is required before a new precommitted three-run trial.

## Protocol correction - August 18, 2026

The three independent pilot runs used this file exactly as written. During review, Codex found that this condensed source says only that Spruce supplies seven pages; it does not name them. The expected evidence state and rubric nevertheless assume Spruce includes all five required pages. The longer design source also contained payment schedules and exclusions that this condensed execution input omitted.

Because a model cannot be scored against facts it was never given, this pilot is invalid for the planned public verdict. The three raw outputs are preserved separately as `TEST_002_PILOT_TRIAL_1.md`, `TEST_002_PILOT_TRIAL_2.md`, and `TEST_002_PILOT_TRIAL_3.md`. They will not be silently substituted, edited, or represented as the official Test 002 runs.

Before rerunning, create a versioned v2 specification containing the complete buyer brief, both complete proposals, the exact prompt, and a rubric whose checkpoints match the supplied text. Then run three new fresh-context trials and publish the first v2 output in full.

## Learning question

Can AI normalize two differently structured vendor proposals, surface material gaps, and preserve uncertainty without manufacturing an apples-to-apples winner?

## Synthetic buyer brief

Lakeview Home Organizing is a synthetic eight-person service business replacing its public website.

Must-haves:

- Home, Services, About, Gallery, and Contact pages.
- Embed the existing Jobber request form.
- Staff can update text and photos.
- Setup plus mandatory charges for the first 12 months after launch stay at or below $7,000 before tax, excluding the existing Jobber subscription and domain registration.

Preferences:

- Go live within six weeks of vendor selection.
- Own and be able to move the final website and content.
- Keep ongoing costs predictable.
- Preference weights have not been assigned.

## Synthetic proposal A - Northline Web Co.

- $6,400 fixed project fee.
- Custom visual design; five required pages; client supplies final copy and images.
- Existing Jobber form embed; no custom Jobber or API integration.
- CMS editor access, one training session, two design-revision rounds, and one pre-launch QA round.
- Estimated six weeks from kickoff, after signed agreement and kickoff payment, assuming feedback within two business days.
- Hosting included for 12 months after launch, then $30/month. Thirty-day implementation-defect correction; later content work $140/hour.
- After final payment, the client receives the final custom design, site files, and approved content. Approved third-party licenses may not transfer.
- Copywriting, photography, logo work, domain, Jobber subscription, custom integration, legal-policy drafting, ongoing SEO, and accessibility audit are excluded.
- Cancellation and refund terms are not stated.

## Synthetic proposal B - Spruce Digital

- $4,200 setup fee plus a required $195/month care plan for 12 months beginning at launch.
- Template adaptation; seven pages; up to 1,500 words of draft copy; client-supplied photos.
- Existing Jobber form embed; no custom data synchronization.
- Editor access, recorded handoff, metadata, XML sitemap, and an unnamed analytics dashboard.
- Estimated four to five weeks after receipt of questionnaire, assets, photos, and cleared first installment.
- Care plan includes hosting, updates, backups, and 60 minutes of monthly content changes; unused time does not roll over. Other work is $125/hour.
- Support target is a response within two business days on weekdays, not a resolution or uptime guarantee.
- Initial-term cancellation requires 50% of remaining monthly payments. After month 12 the plan continues at $195/month, subject to change with 30 days' notice.
- The client owns approved copy and supplied photos. Spruce retains the template, design system, configuration, and source repository; it supplies a content/image export on request after termination.
- Revision count is not stated.

## Exact prompt contract

The comparison agent must:

1. Use only the buyer brief and proposals.
2. Never invent or silently fill a price, feature, deadline, obligation, ownership right, service level, revision allowance, cancellation term, or other missing term.
3. Write `Not stated` for missing terms; absence never means no, zero, free, unlimited, or included.
4. Preserve timeline and price conditions.
5. Separate one-time, mandatory recurring, optional, hourly, renewal, and cancellation costs.
6. Label arithmetic as calculated and show the formula.
7. Distinguish form embed from custom integration, response from resolution, and content ownership from full-site portability.
8. Avoid a weighted score or overall winner because buyer weights are missing.
9. Separate sourced facts, calculations, judgment, and clarification questions.
10. Return requirements check, normalized commercial terms, material differences, risks/unknowns, conditional decision guide, five clarification questions, and confidence/review in under 1,000 words.

## Precommitted run protocol

- Run three fresh-context trials with identical inputs and model settings.
- Publish the first run in full; never substitute the best-looking run.
- Score all three against the same 100-point rubric.
- Record model/version, date, settings, runtime, and cost where available; mark missing measurements unknown.
- Do not edit outputs before scoring. Label any later excerpt as edited and link to the complete output.

## Rubric summary

- Factual fidelity: 30 points across 15 named facts.
- Missing-term discipline: 25 points across five required unknowns.
- Commercial normalization: 15 points, including A = $6,400 and B = $4,200 + (12 x $195) = $6,540.
- Buyer-fit analysis: 15 points.
- Decision usefulness: 10 points.
- Traceability and clarity: 5 points.

Hard fail: inventing a material term; hiding B's recurring charge; treating either timeline as guaranteed from selection; describing the form embed as custom integration; claiming B transfers the complete website; treating response as resolution/uptime; or declaring a winner with invented weights.

## Expected evidence state

- Both proposals meet the baseline page, editor, form-embed, and stated first-year budget requirements.
- A's calculated first-12-month cost is $6,400; B's is $6,540.
- Neither proves launch within six weeks of vendor selection because each schedule starts after later conditions.
- A states stronger ownership and portability.
- B includes copy drafting and continuing managed assistance.
- No unconditional winner is defensible until the owner prioritizes portability versus copywriting/support and receives material clarifications.

## Verdict rule

- Useful with human review: all three runs score at least 90 with no hard fail.
- Promising; revise: no hard fail, but a run scores below 90 or omits a decision-relevant unknown.
- Reject: any hard fail or median score below 75.
- Never ready without review; real proposal claims, contracts, technical fit, and vendor reliability require human verification.

## Privacy

All business names, vendor names, prices, scope details, and terms are synthetic. They are not adapted from an actual proposal, customer, vendor, or private record. A future real-data version must define processing, retention, redaction, and deletion before accepting documents and must never send comparisons or accept contracts automatically.