Datalyze Tracking Agent

Tracking Agent: a PR check for your analytics tracking

The Datalyze Tracking Agent checks every pull request for what it does to your analytics tracking, and holds the merge until a person signs off. It runs on your own GitHub runner with your own AI model, so your code and data never reach Datalyze. It costs $100 a month, or $125 with the Datalyze MCP.

By Ansh AgrawalUpdated

What does the Tracking Agent check on a pull request?

It reads the change and works out four things: tracking the change needs and doesn't have, existing events the change breaks, events that are now dead, and user properties it affects. Breaking an event is the one it treats as worst, because a silently changed event corrupts every dashboard built on it.

  • New behaviour with no event, such as a new checkout step nobody tracks.
  • Broken events: an event renamed in place, a call moved so it fires at a different moment, a property whose values changed.
  • Dead events and stale user properties: calls the change removed, and profile values nothing sets any more.
  • Convention problems: a property that looks like personal data, set where a first-time value needs set_once, an action tracked without its outcome.

Most pull requests need no tracking change. Those pass with a one-line reason, at no model cost when a script can tell, such as a change to styles or docs only.

How does a tracking change get approved?

Nothing reaches your main branch without two decisions from a person: approving the proposal, then signing off on the verified result. The check is a required status check on the pull request, so GitHub holds the merge until both are done.

StepWhoWhat happens
1. ProposeThe agentLists each change with the question it answers, the code it's based on, and whether it breaks anything
2. ApproveYour reviewerApproves, edits or rejects each item with a comment such as /datalyze approve
3. Implement and verifyThe agentUpdates the tracking plan and the code, then runs scripts, your own tests, generated tests and an independent verifier
4. Sign offYour reviewerChecks what each event sends against the plan and signs off. The pull request can merge

If verification fails, the agent tries to fix it twice. After that, or when a fix needs a judgement call, it asks your reviewer and offers lettered options. The Tracking Agent docs explain every status and command.

What does Datalyze see when the Tracking Agent runs?

Datalyze sees that your licence key was used, and when. Everything else stays with you.

WhatWhere it goesDoes Datalyze see it?
Your codeYour GitHub runner, or a laptopNo
Model callsYour provider: Anthropic, OpenAI, Google, AWS Bedrock, Google Vertex, or your gatewayNo
Analytics dataYour analytics tool, read with your own accessNo
Licence checkDatalyze: your key in, the signed method outOnly that the key was used, and when

The method the agent follows is versioned and signed, and the runner checks the signature before using it. You can pin a version, so results never change without you knowing.

How well does the Tracking Agent catch tracking problems?

On our test suite of 13 seeded repos, it caught 32 of the 33 problems we planted across three runs, and left every clean pull request alone. The planted problems are the ones we find most often in client audits: an event renamed in place, a call that now fires at the wrong moment, a missing entry_point, a dead event, a stale user property, set instead of set_once, personal data in a property, and an action with no outcome event.

It also runs on this site's own repo. The first real pull request it checked replaced an email used as a user id. The agent proposed four changes, one of them breaking, then implemented and verified them once they were approved through the pull request.

What does the Tracking Agent cost?

$100 a month for a workspace, or $125 a month with the Datalyze MCP. Both renew monthly until you cancel, and cancelling keeps access to the end of the month you paid for. If you already pay for the MCP, upgrading to both mid-month credits the MCP days you haven't used.

A lapsed payment never stops you shipping. The check then passes with a note that the pull request wasn't checked.

If you don't have a tracking plan yet, start with creating a tracking plan or the free event tracking plan generator. For a one-off clean-up of tracking that has drifted, our analytics audit is the faster route.

Questions about the Tracking Agent

Which analytics tools does the Tracking Agent work with?

The method works with any tool, because the tracking plan lives in your repo. The data snapshot has recipes for Mixpanel, PostHog and Amplitude and a SQL route for BigQuery, Snowflake and ClickHouse; the typed tracker has adapters for Mixpanel, PostHog, Amplitude, Segment, RudderStack and GA4.

Which languages does it support?

TypeScript and JavaScript for web apps, Node servers and React Native, and Swift, Kotlin and Dart for mobile apps.

Does the Tracking Agent need access to our code?

Datalyze never gets your code. The agent runs in your own GitHub Actions runner or on a laptop, and the code it reads goes only to the AI provider you choose, with your own key. The one call to Datalyze sends your licence key.

Do reviewers need a Datalyze account?

No. Only the person who pays needs one. Reviewers approve and sign off with comments on the pull request, and who may do so is set by GitHub access and the reviewers list in your config.

What happens to our pull requests if the subscription lapses?

They pass. The check reports that the Tracking Agent is inactive and the pull request wasn't checked, so a missed payment never blocks a release.