Tracking Agent docs

Which AI models the Tracking Agent can use

The Tracking Agent runs on your own model access: Anthropic, OpenAI, Google Gemini, AWS Bedrock, Google Vertex, your gateway, or a Claude Code login. Nothing goes through Datalyze, and every PR comment ends with the models used, the tokens and the cost. With Anthropic, the defaults are Haiku for triage, Sonnet for the main work and Opus for the verifier.

By Ansh AgrawalUpdated 20 Oct 2026

Which model providers does the Tracking Agent support?

Provider	`model.provider`	Credentials	What else to set
Anthropic	`anthropic`	`ANTHROPIC_API_KEY`	Nothing: defaults are built in
OpenAI	`openai`	`OPENAI_API_KEY`	A model for each role
Google Gemini	`google`	`GEMINI_API_KEY`	A model for each role
AWS Bedrock	`bedrock`	GitHub's OIDC identity; no stored key	`model.region`
Google Vertex	`vertex`	Workload identity federation; no stored key	`model.project` and `model.region`
Your AI gateway	`gateway`	`GATEWAY_API_KEY`	`model.base_url`, `model.gateway_api` (`anthropic` or `openai`) and a model for each role
Claude Code on a laptop	`claude-code`	Your existing Claude Code login	Nothing

Anthropic is the provider the agent has run on for real pull requests and the seeded test suite. The others use the same interface; with them, set model.pricing so reports can state the cost.

Which model does each Tracking Agent step use?

Role	Used for	Anthropic default
`small`	Triage: does this change touch tracking at all?	`claude-haiku-5-5`
`main`	The change map, the four specialists, the critic, the implementer and the test writer	`claude-sonnet-5-5`
`verifier`	The independent verifier, on a different model from the implementer	`claude-opus-5-5`

Change them under model.models in .datalyze/config.yaml. OpenAI, Gemini and gateways have no defaults: name at least main, and any role you leave out uses it.

How do I use an organisation secret for the model key?

If your admin keeps the key as an organisation secret, give setup its name. The workflow passes it to the runner under the provider's usual variable, such as ANTHROPIC_API_KEY, so nothing else changes and no key is copied into the repo.

How do I use AWS Bedrock or Google Vertex without a stored key?

The workflow signs in to your cloud with GitHub's OIDC identity, so there's no long-lived key anywhere. Setup writes this step for you; by hand, it looks like this for Bedrock:

yaml

# .github/workflows/datalyze.yml, in the tracking job, before the Datalyze step
- uses: aws-actions/configure-aws-credentials@v4
  with:
    role-to-assume: arn:aws:iam::123456789012:role/datalyze-bedrock
    aws-region: us-east-1

# .datalyze/config.yaml
model:
  provider: bedrock
  region: us-east-1

For Vertex, the step is google-github-actions/auth@v2 with your workload_identity_provider and service_account, and model.provider: vertex. The workflow already has the id-token: write permission these need.

How do I use an AI gateway with the Tracking Agent?

Point the agent at any gateway that speaks the Anthropic or OpenAI API, such as LiteLLM, OpenRouter or an internal proxy, and save its key as GATEWAY_API_KEY.

yaml

model:
  provider: gateway
  base_url: https://llm.example.com
  gateway_api: anthropic        # or openai
  models:
    small: your-small-model
    main: your-main-model
    verifier: your-verifier-model

How do I run the Tracking Agent on a laptop?

Set model.provider: claude-code and the agent uses your existing Claude Code login. datalyze check --base main then runs the whole PR check on your current branch, with GitHub simulated in .git/datalyze/, and datalyze check --pr <n> runs it against a real pull request.

How is model spend reported and capped?

Every comment ends with the models, tokens and cost. Prices are built in for Anthropic's own API. For Bedrock, Vertex, gateways, OpenAI and Gemini, add model.pricing in US dollars per million tokens (input, output, and optionally cacheRead and cacheWrite), or the cost shows as unknown.

model.max_cost_usd stops a run once its spend passes the cap; the default is $10. Calls whose price isn't known (Bedrock, Vertex, a gateway, OpenAI or Gemini without model.pricing) count against the cap at $5 per million input tokens and $25 per million output tokens, so the cap still stops a run; the report still says their cost is unknown. For scale, our seeded suite of 13 repos costs between $3.32 and $3.65 a run on Anthropic's defaults.

Questions about models

Does Datalyze pay for or proxy model calls?

No. Calls go from your runner or laptop straight to your provider, billed to you. Datalyze never sees the prompts, the code or the answers.

Why does the verifier use a different model?

So it doesn't share the implementer's blind spots. It also starts with a fresh context and none of the first agent's reasoning.

Can we use a model we host ourselves?

Yes, through a gateway that speaks the Anthropic or OpenAI API. Name the model for each role and add its prices under model.pricing.

How do AI agents fit with the rest of our analytics?

The playbook chapter on analytics in the age of AI covers what an agent needs to give right answers, and our AI layer for analytics service builds that for a team's own data.