Sense checking: testing a number before you report it

A sense check is holding a number against what you already know about the business, to see whether it is in the right ballpark before anyone acts on it. Compare it with numbers you have heard, with your own experience, and with a rough rebuild from visits and sign-ups a day. If it fails, verify it before you decide anything on it.

Say you plot your sign-up rate, the share of new website visitors who sign up, and it comes out at 1%. Before you put that number in front of anyone, hold it against what you already know about the business.

That is a sense checkSense checkHolding a number roughly against what you already know about the business, to see whether it is even in the right ballpark, before anyone acts on it.Glossary. You take a number and check it roughly against the business context you already carry in your head, to see whether it is in the right ballpark.

It is one of the most important qualities a founder or PM can have in analytics. And if you have an analyst or a data person working with you, it matters more for them than for anyone.

When you are scaling, you end up with a lot of data and a lot of ways to calculate the same metric, as we covered in the Metric definitions across the board chapter. There are many places where things can go wrong. The data is structured slightly differently, a definition drifts, a filter is off.

So people plot the wrong numbers. And in a good chunk of cases, they look at those numbers and make decisions on them.

The easiest way to avoid that is a sense check.

Back to the 1% sign-up rate. You might remember someone else in the organisation saying, some time back, that the sign-up rate is 10% or 15%. Bring that in and ask the question out loud: someone told me it was 10% or 15%, so are we sure this 1% is right?

Or you might just know, from having worked at companies, that 1% is really low. If we are really at 1%, that is something we need to work on right away.

It runs the other way too. Someone tells you your sign-up rate is 1%, and you think: but I know roughly how many new people come to my website, and I know roughly how many sign-ups we get. If we are doing 1,000 sign-ups a day, a 1% rate means 100,000 new visitors a day. We are not getting anywhere near that.

The examples above come down to three moves.

1. Compare with what you have heard. Has someone quoted a different number before?
2. Compare with your experience. Is it wildly high or low?
3. Rebuild it from rough counts. Use visitors and sign-ups a day to work out what it should be.

To get good at this, keep a few rough numbers about your business in your head, like visits and sign-ups a day. Once you have them, you will catch 80% to 90% of data issues before you show them to anyone else, and before you make a decision on them.

A sense check only catches numbers that look wrong. A number that is wrong but sits inside the range you expected goes through. Those are the ones that do the real damage, because nobody goes looking for them.

It also works in reverse. When something real happens, a channel dies or a release breaks the sign-up flow, the number will look wrong, and your instinct will be to blame the data. Teams lose weeks to that.

So treat the sense check as the first filter, not the last one. It tells you which numbers to go and verify. It does not tell you which ones are correct.

When a number fails, do not decide anything on it yet. Verify it first. The Event verification and Data accuracy as a system chapters cover how.

That matters, because I have seen a lot of founders and PMs look at the wrong number, make a decision on it, and carry on from there.