# GrowthBench Benchmark Charter

## Purpose

GrowthBench is a production benchmark for Growth Agents: systems that can plan,
execute, adapt, and produce verified growth outcomes in a live operating
environment.

This is not a language-model leaderboard. The unit of comparison is the
deployable agent a growth team can actually use.

## Contestants

The first benchmark set is:

- **SITIN-01** — SITINai Growth Agent
- **Fable-5** — Fable Growth Agent
- **Opus 4.8** — Anthropic Growth Agent

Each contestant is evaluated as a black box. Its model, prompts, memory,
tools, policies, execution runtime, and proprietary infrastructure are part of
that agent's capability. GrowthBench does not attempt to normalize away those
differences.

## What Fair Means

GrowthBench is fair at the task and outcome layer, not symmetrical at the
implementation layer.

Every contestant receives the same:

- growth objective and task brief;
- starting cohort and environment state;
- budget, permissions, and time window;
- external platform constraints;
- success metrics, audit rules, and stopping conditions.

The benchmark does not require every contestant to have the same internal
harness. A model API, a bespoke runtime, and a complete production agent are
different products. The customer-facing question is which product produces the
strongest result under the same operating conditions.

SITIN-01 is therefore evaluated with its complete first-party growth runtime,
including its proprietary execution harness. Competitors are evaluated in the
strongest configuration a customer can actually deploy for that contestant;
GrowthBench does not provide them a SITIN-specific domain harness.

This is an outcome comparison between Growth Agents, not a claim that their
underlying models have equal isolated intelligence.

## Operating Loop

Every run follows the same observable contract:

```text
growth objective
  -> agent plan
  -> tool and platform execution
  -> feedback observation
  -> agent adaptation
  -> verified business outcome
```

An agent must be able to do more than produce a recommendation. The run must
capture its actions, recoveries, external effects, and final cohort outcomes.

## Outcome Scorecard

The primary leaderboard is outcome-first. The exact weighting is versioned per
task suite, but every suite reports these dimensions separately:

- aggregate ROAS, defined as verified attributed revenue divided by ad spend;
- verified attributed revenue and contribution margin;
- ad spend and budget efficiency;
- execution completion, recovery, and time to first measurable result;
- account safety and policy-violation rate;
- operator hours and required human intervention.

Composite scores never replace the raw cohort-level measurements. Small-sample
ROAS spikes, isolated purchases, or single-account wins cannot establish a
channel or agent-level result.

## Reproducibility and Audit

- Cohorts and holdout windows are frozen before a scored run.
- The benchmark records the agent version, configuration, interventions,
  retries, spend, and external side effects.
- Runs cannot be selectively stopped after a favorable early result.
- Rewards, approvals, installs, purchases, refunds, and ROAS events
  require a verifiable source of truth.
- Safety failures and account loss remain part of the score; they are not
  hidden as operational noise.

## Positioning

GrowthBench asks:

> **Which Growth Agent can actually grow the business?**

It is a build-vs-buy decision aid for non-AI companies and individuals. The
comparison is not "SITIN-01 versus Claude as a model." It is:

```text
SITIN-01 vs Fable-5 vs Opus 4.8
Which complete agent turns the same growth opportunity into more verified revenue per
unit of ad spend, with durable operating capability?
```

## Public Language

> GrowthBench is an outcome-first benchmark for production Growth Agents.
> Every agent receives the same growth objective, cohort, budget, permissions,
> and deadline. Internal models, memory, tools, runtime, and proprietary
> infrastructure are part of each agent and are not normalized away. We score
> what survives contact with the real world: execution, recovery, safety,
> revenue, and ROAS.

Short version:

> **Models generate possibilities. Growth Agents produce outcomes.**