How to audit an AI-generated query in 60 seconds (three places to look)
The 60-second version
- Read inside out: FROM and JOINs first, WHERE second, GROUP BY third. The SELECT list is cosmetic.
- One join on a date can triple revenue with zero errors — ask whether one row on the left can match several on the right.
- Say the WHERE clause in plain English, compare it with your question, then glance at the byte estimate.
a minute on the joins beats an hour on the chart
You don't need to be fluent in SQL to catch a wrong AI answer. You need a minute and three places to look: the tables it used, the filters it applied, and the level it grouped at. Nearly every confidently wrong number fails at least one of those.
Start with FROM and the JOINs, not SELECT
Most people read a query top to bottom, starting at SELECT. Read it inside out instead, starting at FROM. The select list only decides which columns are shown and what they're called. FROM and JOIN decide which rows the answer is computed over — and a wrong set of rows can't be saved by a nice-looking select list.
Three questions, in order:
Do I recognise these tables? A query answering "revenue last month" from orders_legacy, when the team moved to orders a year ago, is stale before it runs. A query reaching into raw staging tables instead of your cleaned views is improvising. If a table name surprises you, stop — the surprise is the audit doing its job.
Is the join key unique on at least one side? This is the most expensive mistake in AI-written SQL. Join orders to campaign spend on the date, and every order is repeated once for every campaign that ran that day:
-- Wrong: each order counted once per campaign that ran that day.
-- Three campaigns a day = 3x revenue, zero errors.
FROM orders o
JOIN campaign_spend s ON o.order_date = s.spend_date
-- Right: bring each side to one row per day, THEN join.
FROM (SELECT order_date, SUM(net_revenue) AS revenue
FROM orders GROUP BY order_date) o
JOIN (SELECT spend_date, SUM(spend) AS spend
FROM campaign_spend GROUP BY spend_date) s
ON o.order_date = s.spend_dateYou don't have to understand the whole query to spot this. Find each JOIN, read its ON line, and ask: could one row on the left match several rows on the right? If the key is a date, a channel name or a region — yes, and the totals are multiplied.
Does the join match how the business connects these things? Orders link to campaigns through attributed order lines, not the calendar. Spend links to dates directly. Joining the wrong two things on a plausible-sounding key is the model's most characteristic mistake, because the plausible key is what its training data suggests.
Everything else in that query was fine — the filters, the grouping, the formatting. One join line tripled the number.
WHERE must match what you asked
If FROM picks the universe, WHERE picks the slice. Read each filter as an English sentence and hold it up against your question.
The date column. A warehouse usually holds several timestamps per order — created, paid, shipped, delivered — and they answer different questions. "Revenue last month" filtered on shipped_at during a delivery backlog is really "revenue from earlier, shipped late". Also check the range: the last 30 days isn't last calendar month.
The status filter. The classic quiet failure:
-- Silently removes every order whose status is NULL
WHERE order_status != 'cancelled'NULL != 'cancelled' is unknown in SQL, and unknown rows are filtered out — so brand-new orders with no status yet disappear. Safer to list what to include: order_status IN ('paid', 'partially_refunded').
Timezones and division. If the warehouse stores UTC and you report in local time, a day drawn in the wrong zone moves your busiest hours into the wrong day. And SUM(revenue) / SUM(spend) breaks on zero-spend days unless it's wrapped in NULLIF or SAFE_DIVIDE.
GROUP BY must match the noun
The grain — what each output row represents — is where "revenue by campaign" quietly becomes something else.
Does the grouping match the noun in your question? "Customers" means grouping by customer_id. A query grouping by order_id and calling the result "customers" counts repeat buyers once per order, and inflates every ratio built on it.
Is the DISTINCT counting the right thing? COUNT(DISTINCT order_id) and COUNT(DISTINCT customer_id) differ by your repeat-purchase rate. Make sure the thing inside the count is the entity you asked about, not its neighbour.
A useful tell: clean multiples like 2× or 3× usually mean a fan-out join. Numbers that are plausible but off by an awkward factor like 1.3× usually mean the grain is wrong.
The routine
- FROM and JOINs — 25 secondsRecognise every table. For each join, ask whether one row can match several. Dates and names are the usual multipliers.
- WHERE — 20 secondsTranslate each filter to English. Right date column and range, cancellations and refunds handled by inclusion, NULLs safe, timezone converted.
- GROUP BY — 10 secondsThe grouped columns equal the noun in your question; the COUNT DISTINCT targets that same entity.
- Cost and sanity — 5 secondsCheck the dry-run byte estimate — terabytes for a routine question means a missing filter. Then: does the result pass a smell test against last month?
If all four pass, trust the number the way you'd trust a junior analyst's checked work. If any step raises a question you can't settle in the minute, don't ship it — send the SQL to an analyst with the step that failed, or ask the tool to regenerate with a precise correction ("use net_revenue, exclude full refunds, group by customer_id"). A specific correction gets a far better second attempt than "try again".
Quick gut-check
Revenue comes out exactly 3× too high, with no errors. Where do you look first?
The SELECT list
No. SELECT is cosmetic — a wrong set of rows can't be fixed there.
The JOIN keys — one row matching many
Exactly. A join on a non-unique key like a date multiplies every row by the number of matches.
The GROUP BY
Not first. Grain errors give awkward multiples; a clean 3× points at a fan-out join.
Questions people ask
What if I can't read SQL at all?
Read the nouns. Table names after FROM and JOIN, column names in WHERE, and the list after GROUP BY carry most of the audit — and they're words you already know. You can also ask the tool to explain the query one clause per sentence, then audit the sentences.
What if it uses a table I've never heard of?
Treat that as a finding. An unfamiliar table is usually deprecated, raw or staging — ask why it was chosen before trusting anything built on it.
Does this catch every mistake?
No. It catches translation mistakes — the query answering a different question. Stale pipelines, missing sources and definitions the team disagrees on need monitoring, source checks and a shared metric definition.
Takeaways
- Read inside out: FROM and JOINs, then WHERE, then GROUP BY.
- On every join, ask whether one row can match several.
- Say the WHERE clause in English and compare it with your question.
- GROUP BY should equal the noun; the DISTINCT should count that same thing.
- Finish with the byte estimate and a smell test — and escalate with the step that failed.