Tracking Agent docs

How to set up the Tracking Agent

Set up the Tracking Agent in the browser: buy a plan, connect GitHub, choose model and analytics access, set the rules, then install. Installing opens a setup pull request, and the install audit runs on it and drafts your tracking plan from the code. datalyze init runs the same steps in a terminal.

By Ansh AgrawalUpdated

What do I need before setting up the Tracking Agent?

  • A Datalyze account with the Tracking Agent or the bundle. Only the person who pays needs one; reviewers sign in with GitHub.
  • Admin access to the GitHub repo, so setup can save secrets, add the workflow, create a GitHub App and make the checks required.
  • Model access: an API key, an organisation secret, a cloud identity on AWS Bedrock or Google Vertex, or an AI gateway. See models.
  • A TypeScript, JavaScript, Swift, Kotlin or Dart codebase. Mobile apps need a little more config: see mobile apps.
  • Optional: read access to your analytics tool, and a Google Sheet.

How do I set up the Tracking Agent in the browser?

  1. Buy a plan. Sign up or sign in, then choose the Tracking Agent ($100 a month) or the bundle with the Datalyze MCP ($125 a month).
  2. Create a workspace and connect GitHub. Sign in with GitHub and pick the repos to check.
  3. Choose model access: a provider and key, the name of an organisation secret, a cloud identity, a gateway, or set it up yourself later.
  4. Choose analytics access: your tool and a read-only key or service account. Mixpanel, PostHog and Amplitude have recipes; see connecting analytics. For a warehouse, also say where the events are and which SDK your app sends them with.
  5. Connect a Google Sheet (optional): copy the Datalyze template and paste its receiver URL and token, or use a Google service account. See the Google Sheet.
  6. Set the rules: reviewers and signers (GitHub logins or teams written @org/team), whether pull request authors may approve their own tracking, and your platforms. For a mobile app, also how you cut a release, so the post-release watch knows which version ships a change. For a repo in an organisation, setup gives your App read access to the organisation's members, so team entries work.
  7. Install. Setup creates your own GitHub App, saves every secret to the repo encrypted in your browser, opens the setup pull request, and makes datalyze/tracking and datalyze/rules required checks on your default branch, accepted only from your App.
  8. Read the install audit. It runs on the setup pull request and commits the drafted registry, the baseline, the typed tracker and a report to it. Merge once you've read the report.

Setup starts from the Datalyze default conventions: snake_case object_action event names such as post_created. Change them in analytics/registry/_conventions.yaml, in the setup pull request or later. The playbook chapter on creating a tracking plan covers choosing a naming convention.

What does the setup pull request add?

FileWhat it is
.github/workflows/datalyze.ymlThe workflow. It runs on pull requests, on /datalyze comments, after each merge to your default branch (to rebuild the Sheet), every hour (for the Sheet's Requests tab) and on a manual run, and posts both statuses
.datalyze/config.yamlHow the check runs in this repo: model, reviewers, your own checks, analytics, the Sheet
.datalyze/README.mdWhat each file is, the comment commands, and a link to these docs
.datalyze/.gitignoreKeeps the cached method out of git
analytics/registry/_conventions.yamlYour naming rules and reviewers, merged over the Datalyze defaults

The install audit then adds the rest of the registry, _baseline.json, the typed tracker in analytics/generated/ and a report, as commits on the same pull request. Nothing in your product code changes.

When a call builds its event name at runtime, such as from a map of steps to names, and each name appears as a string in the code, the install audit records those names as scan.dynamic_names, so the events reach the registry. Properties a wrapper adds to every event on the client or the server are declared once, as super properties.

The install audit runs only on the first setup pull request: a pull request from the branch datalyze/setup gets it when its base branch has no .datalyze/config.yaml yet. After setup has merged, a branch with that name is an ordinary pull request and gets the normal check.

Which secrets does Tracking Agent setup save?

SecretWhat it holdsWhen
DATALYZE_KEYYour workspace's Tracking Agent key, from your Datalyze dashboard. It also signs the state the check keeps on each pull requestAlways
DATALYZE_APP_ID, DATALYZE_APP_PRIVATE_KEYYour own GitHub App, so your CI runs on the agent's commitsRecommended
ANTHROPIC_API_KEY, OPENAI_API_KEY, GEMINI_API_KEY or GATEWAY_API_KEYYour model keyWhen the key is a repo secret
MIXPANEL_SA_USERNAME, MIXPANEL_SA_SECRETA Mixpanel service account with read accessWith Mixpanel
DATALYZE_SHEET_URL, DATALYZE_SHEET_TOKENYour Sheet's receiver and its tokenWith the Sheet template
GOOGLE_SERVICE_ACCOUNT_JSONA Google service account key with edit access to the SheetWith a service account instead

Each value is encrypted in your browser with the repo's public key and sent straight to GitHub, so none of them reaches Datalyze's server. GitHub explains how secrets in GitHub Actions are stored and who can read them.

How do I set up the Tracking Agent from the command line?

datalyze init asks the same questions in the terminal and writes the same workflow, config, conventions and README, saves the same secrets, and opens the same setup pull request. Run in a mobile app's checkout, it also finds the app and proposes its typed tracker and generated unit tests. Run datalyze init --help for its options.

bash
npx datalyze init

To set up by hand instead: datalyze workflow --action <owner/repo/path@ref> writes .github/workflows/datalyze.yml, and datalyze github-app create creates your GitHub App and saves its two secrets. [ANSH: confirm the CLI's install command once the package is published]

Why does setup create a GitHub App in my account?

Commits pushed with the default Actions token don't start other workflows, so your CI would never run on the agent's commit and its required checks would wait forever. GitHub documents this in automatic token authentication. Commits pushed with your own App's token do start them.

The App belongs to your account, is installed only on the repos you choose, and its private key lives in your repo's secrets. Without it the agent still works, and its report says your CI won't run on its commits.

The App also decides who can post the required checks. With it, the ruleset accepts datalyze/tracking and datalyze/rules only from your App; without it, any token with write access to the repo could post them.

One risk is left either way. Anyone who can push a branch that changes .github/workflows/ can run code with the repo's secrets on that pull request, the App's key included. Protect .github/workflows/ and .datalyze/ with CODEOWNERS and turn on required code-owner review for your default branch, so a change to either can't merge without its owner's review, and give write access only to people you'd trust with those secrets.

Questions about setting up the Tracking Agent

Does Tracking Agent setup change our product code?

No. The setup pull request adds a workflow, the .datalyze folder and the registry. The typed tracker it generates isn't used until you move call sites to it in reviewed pull requests.

Who needs admin access during setup?

Only the person running setup: saving repo secrets, adding the required checks and installing a GitHub App all need admin rights. Reviewers need only their normal access to the repo.

We already have a tracking plan in a spreadsheet. Can setup import it?

Yes, from the command line, and browser setup points you to it. datalyze import-sheet --csv plan.csv,purchase.csv reads CSV exports of the old tabs (or --tabs "Tracking Plan" reads them from a connected Sheet), writes the registry, keeps the old verdicts and comments on each event for review, and lists disagreements between tabs in _import_conflicts.md.

Can Datalyze build our tracking before we add the check?

Yes. If the tracking itself needs designing or rebuilding, our analytics setup service does that work, and the Tracking Agent keeps it correct afterwards.