# AI vendor scorecard

Assess how a partner will deliver and maintain your process, beyond the quality of a demonstration. Use the same questions and trial case for every proposal.

Revision: 2026-09-24

## When to use it

When comparing consultants, agencies or AI solutions before choosing an implementation and ongoing support partner.

## Expected output

An evidence-based comparison with explicit conditions and gaps to clarify before a decision.

## 1. Align the scope

Proposals are not comparable when they cover different work. Share an identical brief and ask each supplier to state exclusions and dependencies.

### Outcome, systems and deliverables

List the process, integrations, users, materials and activities each proposal should cover.

________________

### Business and supplier responsibilities

Specify who prepares data, access, tests, training and operational approvals.

________________

### Exclusions and open conditions

Record licences, custom work, environments and limits absent from the proposal.

________________

> Illustrative example, replace with your own: Illustrative example: two proposals include an assistant, but one excludes source updates and maintenance. Request equivalent operating scope before comparing them.

## 2. Request relevant evidence

A reference and a trial on your own case answer different questions. Preserve verifiable evidence rather than inferring capabilities from logos or broad claims.

### Trial and acceptance criteria

Define representative cases, critical failures, review effort and access to detailed results.

________________

### Delivery and operating approach

Ask how versions, permissions, exceptions, monitoring and changes in dependencies are managed.

________________

### Support and available skills

Record actual delivery roles, agreed coverage, escalation and who will help after release.

________________

> Illustrative example, replace with your own: Illustrative example: ask for a case whose answer is absent from the sources. Check whether the system acknowledges this and how the team handles the exception.

## 3. Compare cost and continuity

Use the same time horizon and volumes for costs. Treat ownership, exportability and supplier replacement as questions to settle in the applicable documents.

### Initial, recurring and variable costs

Separate implementation, licences, usage, support, changes and internal effort; identify estimates.

________________

### Access, documentation and exit

List what must remain accessible: data, configuration, agreed code and operating instructions.

________________

### Evidence, gaps and decision

Record assessment by criterion, evidence links, unresolved clarifications and the reason for the choice.

________________

> Illustrative example, replace with your own: Illustrative example: choose a proposal after clarifying access ownership and change costs. A lower initial amount alone does not resolve these differences.

## Pitfalls to avoid

- Comparing price before making scope equivalent.

- Accepting a selected demonstration as proof of reliability on your data.

- Leaving support, access to materials and exit arrangements implicit.

Stolen Orbit — https://stolenorbit.com/en/templates/ai-vendor-scorecard/