# Pilot Scorecard

Daily and weekly tracking of accuracy, adoption, overrides and feedback during a department pilot, ending in a go, extend or stop decision.

## What this is for

The pilot is where a Clone meets real work with a small group: the champion plus three to five users, with a person in the loop on every output. This scorecard turns two weeks of daily use into a clear decision. It tracks accuracy, adoption, overrides, time saved and user feedback against thresholds agreed before the pilot started. "Pilot accuracy and adoption above thresholds" is the exit criterion for this stage; the scorecard is the proof.

## When to use it

Set thresholds before the pilot starts. Update daily during the pilot. Hold the decision review at the end of Week 5. Keep the scorecard: it is the first evidence for the Clone's next autonomy promotion.

## Who's involved

- **Champion:** leads the pilot group, collects feedback daily.
- **Pilot users (3–5):** use the Clone on real work, flag issues.
- **Implementation Engineer:** fixes issues, reruns the test pack after changes.
- **Business owner:** decides go, extend or stop.
- **InfoSec lead:** reviews exceptions and any incidents.
- **Project Clone:** compiles the daily numbers and weekly summary.

## How to fill it in

1. Before Day 1, agree thresholds for each measure in Section 2 with the business owner and InfoSec.
2. Confirm every workflow is running at its approved pilot autonomy level, usually Assist or Recommend.
3. Each day, record volume, accuracy, overrides and issues. The champion runs a 10-minute check-in with pilot users.
4. Log every issue in the issue log. Fix, rerun the test pack, and note the version.
5. At the end of each week, complete the weekly summary.
6. At the decision review, compare results against thresholds and record the decision.

## The template

### 1. Pilot setup

| Field | Entry |
|---|---|
| Department / Clone | |
| Workflows in pilot | |
| Autonomy level during pilot | _per workflow_ |
| Champion | |
| Pilot users | |
| Pilot dates | _from – to_ |
| Test pack run passed | _Run # and date_ |
| Baseline reference | _link to Success Metrics & Baseline_ |

### 2. Thresholds (agreed before Day 1)

| Measure | Definition | Threshold to go live | Agreed threshold |
|---|---|---|---|
| Accuracy | % of outputs acceptable without major edits | ≥ 90% | |
| Critical errors | Errors that would have caused a prohibited action, data exposure or financial loss if not caught | 0 | |
| Override rate | % of Clone outputs rejected or substantially changed | ≤ 15% | |
| Adoption | % of eligible items where pilot users used the Clone | ≥ 70% | |
| Time per item | Average handling time vs. baseline | Better than baseline | |
| User confidence | Pilot users answering 4 or 5 to "I would rely on this Clone for this work" (1–5) | ≥ 4 of 5 users | |
| Escalation correctness | % of items that should have escalated and did | 100% | |

### 3. Daily log

| Day | Date | Items handled | Used Clone | Acceptable without major edits | Overrides | Critical errors | Escalations (correct / missed) | Avg time per item | Issues logged | Champion note |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | | | | | | | / | | | |
| _4_ | _wk 4 Thu_ | _142_ | _118_ | _109_ | _9_ | _0_ | _6 / 0_ | _8.5 min_ | _2_ | _Two invoices with handwritten PO numbers misread; logged_ |

### 4. Issue log

| ID | Day | Workflow | What happened | Severity | Root cause | Fix | Test pack rerun | Status |
|---|---|---|---|---|---|---|---|---|
| PI-01 | | | | Critical / High / Medium / Low | Instructions / Knowledge / Skill / Connection / Data / Training | | Run # | Open / Fixed / Won't fix |

### 5. Weekly summary

| Measure | Threshold | Week 1 | Week 2 | Trend | Met? |
|---|---|---|---|---|---|
| Accuracy | | | | | |
| Critical errors | | | | | |
| Override rate | | | | | |
| Adoption | | | | | |
| Time per item | | | | | |
| User confidence | | | | | |
| Escalation correctness | | | | | |
| _Example: Accuracy_ | _≥ 90%_ | _88%_ | _94%_ | _Up_ | _Yes_ |

**What users said (quotes):**
- _"..."_

**Top three fixes made:**
1. _..._

### 6. Autonomy readiness view

Record, per workflow, whether the pilot evidence supports staying at the current level or planning a promotion after launch.

| Workflow | Pilot level | Evidence so far | Recommendation | Earliest promotion review |
|---|---|---|---|---|
| | Assist / Recommend | | Stay / Plan promotion to ___ | |
| _Match invoice_ | _Recommend_ | _94% accepted wk 2; 0 critical; override 6%_ | _Plan promotion to Execute with approval_ | _4 weeks after launch_ |

### 7. Decision

| Option | When to choose it |
|---|---|
| **Go** | All thresholds met, no open Critical or High issues |
| **Go with conditions** | Thresholds met on core workflows; named workflows stay in pilot or at a lower level |
| **Extend** | Close to thresholds and trending up; fixes in progress (extend by one week, once) |
| **Stop** | Critical errors persist, or adoption is low for reasons a fix won't solve; return to Design |

**Decision:** Go / Go with conditions / Extend / Stop
**Conditions:** _____________
**Launch date:** _____________

## Good looks like

- **Thresholds set before Day 1.** Moving the bar after seeing results undermines trust in every later promotion.
- **Daily, not just weekly.** Small fixes on Day 2 turn a borderline pilot into a clear go.
- **Adoption and accuracy together.** High accuracy with low use usually means a workflow or training problem.
- **Quotes from users,** good and bad. They become launch communications and training material.
- **Stop is a valid outcome.** A stopped pilot that returns to Design is cheaper than a failed launch.
- **Common mistake:** a pilot group of enthusiasts only. Include at least one skeptical user.

## How Clone helps

The **Project Clone** pulls usage, overrides and escalations from the platform each day, compiles the daily log and weekly summary, and posts a short status to the champion and business owner. The **Enablement Clone** collects user feedback and answers how-to questions during the pilot. In the portal, the scorecard is the Pilot Scorecard deliverable for the department's rollout; the decision is the Pilot exit criterion and the scorecard is linked to the Clone's first autonomy promotion request.

## Sign-off

| Role | Name | Date | Decision |
|---|---|---|---|
| Business owner | | | Go / Go with conditions / Extend / Stop |
| Champion | | | Agree / Disagree (with reason) |
| InfoSec lead | | | No open security issues |

## Related

- [Test Scenario Pack](/playbook/templates/test-scenario-pack)
- [Go-Live Checklist](/playbook/templates/go-live-checklist)
- [Autonomy Promotion Evidence Pack](/playbook/templates/autonomy-promotion-evidence-pack)
- [Success Metrics & Baseline](/playbook/templates/success-metrics-baseline)
- [Give your clone its first task](https://support.clone.org/articles/give-your-clone-its-first-task)

---
From the Clone Solutions Playbook: https://solutions.clone.org/playbook/templates/pilot-scorecard
