Seven questions to ask before you trust an AI's answer about your data (before you paste it into the deck)

An AI can write a query that runs in under a second, throws no errors, returns a neatly formatted number — and still be answering a different question from the one you asked.

That's the failure worth worrying about. When code breaks, it breaks loudly: red text, nothing returned. When a query is semantically wrong, nothing tells you. You get a confident number and a clean chart.

Before an AI-generated number goes into a slide for your leadership team, run these seven checks. None of them needs you to write SQL — only to read a little of it.

1. Can you see the query at all?

The first check is yes or no: did the tool show you the exact SQL it ran, or just a number? If you can't see the query, you can't check whether the logic matches how your company defines the metric. A tool that shows the SQL behind every number lets anyone audit it in seconds. One that doesn't is asking for faith.

2. Which table did it read from?

"Revenue" rarely lives in only one place. You might have:

• orders — order totals including tax and shipping,
• order_items — item prices before and after discounts,
• payouts — what actually settled after payment-gateway fees.

-- What the AI wrote: includes tax and shipping
SELECT SUM(order_total) FROM orders WHERE status = 'paid';

-- What finance means by revenue: net item value after discounts
SELECT SUM(item_price - item_discount) FROM order_items;

The check: look at FROM first. Is that the table your team treats as the source of truth for this metric?

3. What is WHERE really filtering?

This is where "ran successfully" and "right number" part ways most often. The classic case is excluding cancellations:

-- Looks careful. Quietly drops every order whose status is still empty.
SELECT SUM(order_total)
FROM orders
WHERE status != 'cancelled';

In SQL, NULL != 'cancelled' isn't true — it's unknown — so brand-new orders with no status yet vanish from the total. Safer: name what to include, e.g. status IN ('paid', 'partially_refunded').

While you're here, confirm the date column. "Created", "paid" and "shipped" timestamps answer different questions, and "last 30 days" isn't "last month".

4. Could a join be multiplying rows?

A join on a key that isn't unique is the most common reason an AI number comes out mysteriously 1.3×, 2× or 5× too high.

Ask for "revenue and ad spend by day" and a model may join orders straight to the spend table on the date. If four campaigns ran on Tuesday, every Tuesday order now appears four times:

-- Every order is repeated once per campaign that ran that day
SELECT o.order_date, SUM(o.order_total) AS revenue
FROM orders o
JOIN campaign_spend s ON o.order_date = s.spend_date
GROUP BY o.order_date;

The check: for each join, ask "could one row on the left match several rows on the right?" If yes, each side should be summed to one row per day before joining.

5. Does the grain match the question?

The GROUP BY decides what each row of the answer represents. "Customers" should be grouped by customer_id — not order_id, which counts a repeat buyer once per order and inflates everything built on top of it, from CAC to repeat rate.

6. Would someone who knows the business agree?

Turn the query into one plain sentence:

"It took every row in the signups table from last week, counted them whether or not the email was verified, and divided by total site visits."

Read that to your growth lead. If they say "we only count verified work emails", you just caught the problem before leadership did.

7. Zeros, timezones and refunds?

Models assume tidy data. Real data isn't:

• Zero-spend days — dividing by spend needs NULLIF or SAFE_DIVIDE, or the query breaks or misleads.
• Timezones — if the warehouse stores UTC and the business reports in IST, day boundaries move the busiest evening hours into the wrong day.
• Refunds — negative rows in the same table, or positive rows in a separate one? The query needs to know.

SELECT
  DATE(created_at, 'Asia/Kolkata')                         AS ist_day,
  ROUND(SUM(net_revenue) / NULLIF(SUM(ad_spend), 0), 2)     AS roas
FROM daily_metrics
GROUP BY ist_day;

The pocket version

• See the SQL — no query, no trust.
• Check the FROM — is it your source of truth for this metric?
• Read WHERE in plain words — right date column, cancellations and refunds handled, NULLs safe.
• Question every join — can one row match many?
• Match GROUP BY to the noun — customers means customers, not orders.
• Translate it for an expert — let them object early.
• Assume messy data — zero guards, timezones, refund signs.

Quick gut-check