# Proof Before Prompt — Test 002 Specification v2 ## Version note This v2 specification corrects the invalid pilot input, which omitted the named pages, payment terms, and exclusions required to support the precommitted rubric. This is the authoritative, self-contained execution specification. Do not use the condensed `TEST_002_DRAFT` as model input. ## Test identity **Test 002 · Purchasing** **Workflow:** Two website proposals → decision brief **Learning question:** Can AI normalize two differently structured vendor proposals, surface material gaps, and preserve uncertainty without manufacturing an “apples-to-apples” winner? **Audience value:** Owners of 1–20 person service businesses often compare proposals infrequently and under time pressure. The useful job is not “pick a vendor for me”; it is “show me what is actually stated, what differs, and what I still need to ask.” ## Exact execution input Run each trial in a fresh context using the entire block below, unchanged. ```text You are preparing an audit-friendly vendor comparison for a small service-business owner. Use only the Buyer Brief and Proposal A and Proposal B below. Do not use outside knowledge. Rules: 1. Never invent, estimate, or silently fill in a price, feature, deadline, obligation, ownership right, service level, revision allowance, cancellation term, or other missing term. 2. If a proposal does not state something, write "Not stated." Absence does not mean no, zero, included, excluded, free, unlimited, or equivalent. 3. Preserve every condition attached to a price or timeline. Do not compare durations without retaining their different starting triggers. 4. Keep one-time fees, mandatory recurring fees, optional fees, and hourly rates separate. 5. You may perform arithmetic only from stated figures. Label calculated values "Calculated," show the formula, and state what is excluded. 6. Do not treat an embedded Jobber form as custom integration or data synchronization. 7. Do not treat a support response target as a resolution guarantee or uptime commitment. 8. Distinguish ownership of copy and images from ownership or portability of the full website, template, configuration, and source files. 9. Do not create a weighted score or overall winner because the buyer has not supplied preference weights. 10. Separate sourced facts, calculations, judgment, and clarification questions. Return these sections: A. REQUIREMENTS CHECK A table with: - Buyer requirement or preference - Proposal A evidence - Proposal B evidence - Status for each vendor: Meets, Does not meet, or Unclear - Short supporting source phrase B. NORMALIZED COMMERCIAL TERMS Show: - Setup fee - Mandatory recurring charges - Calculated first-12-month cost - Renewal cost after month 12 - Optional or out-of-scope rates - Payment schedule - Cancellation terms - Any commercial term that remains unknown C. MATERIAL SCOPE AND SERVICE DIFFERENCES List only decision-relevant differences. Do not label a difference better or worse without explaining the buyer preference that would make it so. D. RISKS AND UNKNOWNS Separate vendor-specific unknowns from risks directly supported by stated terms. E. CONDITIONAL DECISION GUIDE Give no more than three "Choose A if..." or "Choose B if..." statements. If the evidence does not support an overall winner, say so directly. F. CLARIFICATION QUESTIONS Give the five highest-value questions, addressed to the appropriate vendor. G. CONFIDENCE AND REVIEW State what the comparison can establish, what it cannot establish, and what a human should verify before signing. Keep the answer under 1,000 words. --- BUYER BRIEF --- Lakeview Home Organizing is a synthetic eight-person service business replacing its public website. Must-haves: - Home, Services, About, Gallery, and Contact pages - Embed the company’s existing Jobber request form - Staff can update text and photos - Setup plus mandatory charges for the first 12 months after launch must stay at or below $7,000 before tax, excluding the existing Jobber subscription and domain registration Preferences: - Go live within six weeks of vendor selection - Own and be able to move the final website and content - Keep ongoing costs predictable The owner has not assigned weights to the preferences. --- PROPOSAL A --- Northline Web Co. — Custom Website Project Project fee: $6,400 fixed Scope: - Discovery call and sitemap - Custom visual design - Five pages: Home, Services, About, Gallery, and Contact - Migration of client-supplied final copy and images for those five pages - Embed the client’s existing Jobber request form - Responsive implementation - Google Analytics 4 installation - Page titles, meta descriptions, and XML sitemap - Content-management-system editor access - One 60-minute staff training session - Two consolidated design revision rounds - One pre-launch quality-assurance round Schedule: Delivery is estimated at six weeks from kickoff. Scheduling begins after Northline receives the signed agreement and kickoff payment. The estimate assumes consolidated client feedback within two business days of each review request. Payment: - 40% at signing - 40% at design approval - 20% before launch Hosting and support: Managed hosting is included for the first 12 months after launch. It renews month-to-month at $30 per month after that. Northline will correct implementation defects reported within 30 calendar days after launch. Later content-update assistance is available at $140 per hour. Ownership: After final payment, Northline transfers the final custom design, site files, and approved site content to the client. Third-party software licenses, if any are later requested and approved, are not included in the project fee and may not be transferable. Exclusions: Copywriting, photography, logo work, domain registration, the Jobber subscription, custom Jobber or API integration, legal-policy drafting, ongoing SEO, and a formal accessibility audit. --- PROPOSAL B --- Spruce Digital — Website Launch + Care Plan Setup fee: $4,200 Required care plan: $195 per month for 12 months beginning on the launch date. The first charge is made at launch, followed by 11 monthly charges. Scope: - Adaptation of a Spruce website template - Seven pages: Home, Services, About, Process, Gallery, FAQ, and Contact - Up to 1,500 words of draft website copy based on a client questionnaire - Client-supplied photos - Embed the client’s existing Jobber request form; no custom data synchronization - Responsive implementation - Basic page metadata and XML sitemap - An analytics dashboard - Editor access - One recorded 45-minute handoff session Schedule: Delivery is estimated at four to five weeks after Spruce receives the completed questionnaire, brand assets, client photos, and cleared first setup installment. Payment: - 50% of the setup fee at signing - 50% of the setup fee at launch - Care-plan payments billed monthly beginning at launch Care plan: The care plan includes hosting, security updates, daily backups, and up to 60 minutes of content changes per month. Unused time does not roll over. Other work is billed at $125 per hour. Spruce targets a response within two business days for support requests submitted Monday through Friday. The care plan may be cancelled during its initial 12-month term by paying 50% of the remaining monthly payments. After the initial term, it continues month-to-month at $195 per month. Spruce may change that rate with 30 days’ notice. Ownership: The client owns final approved copy and client-supplied photos. Spruce retains its template, design system, software configuration, and source repository. At termination, Spruce will provide an export of the current page copy and uploaded images on request. The template license and source repository are not transferred. Exclusions: Domain registration, the Jobber subscription, custom Jobber integration, photography, logo work, legal-policy drafting, and a formal accessibility audit. ``` ## Precommitted run protocol - Run three fresh-context trials with identical inputs and model settings. - Publish the first run in full; never substitute the best-looking run. - Score all three independently against the same rubric. - Record model/version, date, settings, runtime, and cost where instrumented. Mark unavailable measurements as unknown. - Do not edit outputs before scoring. Any cleaned representative excerpt must be labeled as edited and linked to the complete output. ## Precommitted rubric — 100 points ### 1. Factual fidelity — 30 points Award two points each for correctly preserving these 15 facts: 1. A’s setup fee is $6,400. 2. B’s setup fee is $4,200. 3. B’s required plan is $195 monthly for 12 months. 4. A’s first 12 months of hosting are included. 5. A renews at $30 monthly after month 12. 6. B continues at $195 monthly after its initial term, subject to possible rate change. 7. A supplies five pages; B supplies seven. 8. A requires client-supplied final copy; B drafts up to 1,500 words. 9. Both embed the existing Jobber form. 10. Neither proposal supplies custom Jobber integration. 11. A estimates six weeks from kickoff. 12. B estimates four to five weeks after receipt of named materials and payment. 13. A transfers the specified site assets after final payment. 14. B retains the template, configuration, and source repository. 15. A and B provide materially different post-launch services. No partial credit within a checkpoint. ### 2. Missing-term discipline — 25 points Award five points each if the output explicitly preserves these unknowns: 1. A’s cancellation and refund terms are not stated. 2. B’s number of revision rounds is not stated. 3. Neither proposal confirms elapsed time from vendor selection to launch. 4. A states no ongoing response target or uptime commitment; B states no resolution guarantee or uptime commitment. 5. B does not name the analytics platform. Unsupported claims elsewhere receive zero for this section and may trigger a hard fail. ### 3. Commercial normalization — 15 points - A first-12-month calculation: **$6,400** — 5 points - B first-12-month calculation: **$4,200 + (12 × $195) = $6,540** — 5 points - Correctly separates renewal, hourly, optional, and cancellation charges — 5 points The $140 difference may be stated, but it must not be used alone to call A “cheaper overall.” ### 4. Buyer-fit analysis — 15 points - Recognizes both proposals cover the five required pages, editor access, and form embed — 4 points - Recognizes both fit the stated first-year ceiling on stated mandatory fees — 3 points - Marks the selection-to-launch preference unclear for both — 4 points - Accurately distinguishes A’s stronger stated portability from B’s retained platform assets — 4 points ### 5. Decision usefulness — 10 points - Identifies the real tradeoff: custom ownership/control versus included copy and ongoing managed assistance — 4 points - Gives genuinely conditional guidance rather than declaring a universal winner — 3 points - Asks five questions capable of changing the decision — 3 points ### 6. Traceability and clarity — 5 points - Clearly separates sourced terms, calculations, judgments, and unknowns. - Preserves short source phrases sufficient to audit material conclusions. ## Hard-fail conditions Any one produces a **Reject for this workflow** verdict: - Inventing a material price, commitment, feature, right, or missing term - Treating an unstated term as free, included, absent, or unlimited - Calling B’s $4,200 setup fee its total first-year cost - Calling either timeline guaranteed from vendor selection - Describing the form embed as a custom Jobber integration - Claiming B transfers the complete website or source repository - Presenting response time as resolution time or uptime - Declaring an overall winner using preference weights the buyer never supplied ## Expected failure traps - Anchoring on B’s lower setup fee while hiding its mandatory recurring charge - Treating A’s included first-year hosting as free forever - Comparing “six weeks” with “four to five weeks” while dropping their different triggers - Equating more pages with better scope despite the different copy and design models - Conflating a form embed with integration - Treating A’s 30-day defect correction and B’s care plan as equivalent support - Calling B’s two-business-day response target a resolution guarantee - Assuming B’s analytics dashboard is GA4 - Assuming B offers unlimited revisions - Assuming A includes copywriting - Conflating ownership of content with ownership of the whole site - Calculating B’s early-termination amount without a cancellation month - Ignoring A’s third-party-license transfer exception - Recommending a vendor without obtaining the owner’s tradeoff priorities ## Expected evidence-based decision state This is the answer key, not a prewritten model verdict: - Both proposals satisfy the baseline page, editor-access, form-embed, and stated first-year budget requirements. - A’s calculated first-12-month cost is $6,400. - B’s calculated first-12-month cost is $6,540. - Neither proposal proves it can launch within six weeks of vendor selection because each schedule starts after a later condition. - A provides stronger stated ownership and portability. - B provides copy drafting and continuing managed assistance that A does not include. - No unconditional winner is defensible until the owner prioritizes portability against copywriting and ongoing support and receives answers to the material clarification questions. ## Honest verdict framework - **Useful with human review:** All three runs score at least 90, with no hard fail. - **Promising; revise:** No hard fail, but any run scores below 90 or omits a decision-relevant unknown. - **Reject for this workflow:** Any hard fail occurs, or the median score is below 75. - **Never “ready without review”:** Proposal claims, contract language, technical fit, and vendor reliability still require human verification. The public verdict should say the test measures whether AI can create a disciplined comparison from supplied text. It does not validate either vendor, establish market pricing, interpret a contract, or prove reliable behavior beyond the observed runs. ## Privacy statement All business names, vendor names, prices, scope details, and contract terms in this test are synthetic and were written solely for the experiment. They are not adapted from an actual proposal, customer, vendor, or private business record. No personal, confidential, or proprietary information is used. A real-business version should not accept proposal documents until the operator confirms they may be processed in the selected environment. Contact details, signatures, account information, confidential pricing, and embedded credentials should be removed or minimized. The comparison must never be sent to a vendor or used to accept a contract automatically.