Anyone can ask the data now. Who keeps the numbers right?
AI made asking easy. Connect a model to the warehouse and you get an answer in a minute. This edition is about what has to be true before anyone acts on that answer: the argument, the research behind it, the KPIs it applies to, and a walkthrough of Bicycle doing the work.
Why can't I just do this with Claude?
It's the question every data leader is being asked, and the demo is real: Claude writes the SQL, runs it and explains the result. The risk is the number that looks right. A join that counts some orders twice, a holiday exclusion applied to every week, last year's definition of an active customer. Our new post walks through what has to be true around Claude before anyone should act on the answer, and which parts of the job suit a language model and which don't.
Read the postOnly one of those five is a natural job for a language model.
- Semantics defines the number.People, in a governed semantic layer
- Statistics finds the signal.SQL, statistics and machine learning
- The model ranks explanations.A language model using tools
- Policy bounds the action.Rules and an approval workflow
- Verification measures the outcome.Statistics, plus human review
Accuracy comes from the definitions, not the model.
Anthropic reports that without its curated reference files, Claude's ability to answer analytics questions accurately "didn't exceed 21%" on its own evals; with them, accuracy is "consistently above 95% in aggregate." When nobody kept those definitions current, accuracy drifted "from ~95% at launch to ~65% over a month." In a peer-reviewed study, researchers at GSK found GPT-4's accuracy rose "from 8.3% with the database schema alone to 78.3% with the business context document."
Our own benchmark points at the cost of finding out late. In Bicycle's 2026 survey of 158 product and analytics leaders at large retail and ecommerce companies, 45.6% said customers report revenue-impacting issues before their own tools detect them most of the time, and another 5.7% said always.
Read the 2026 retail and ecommerce benchmarkSame model. Different context.
Accuracy before and after the business definitions were added
Claude on Anthropic's analytics evals
GPT-4 in GSK's peer-reviewed study
| Claude on Anthropic's analytics evals, without curated reference files | did not exceed 21% |
|---|---|
| Claude on Anthropic's analytics evals, with them | consistently above 95% |
| GPT-4 in GSK's study, with the database schema alone | 8.3% |
| GPT-4 in GSK's study, with the business context document | 78.3% |
Six KPIs that moved. How Bicycle found why.
Each is a use case from our catalogue. Open one to see the KPI, where it broke, and how Bicycle found the cause and recommended the next step.
- Availability by regionStockouts hiding in one regionAvailability looks fine overall. So why does one region keep running out?Read the use case
- Search conversion by keyword and locationSearch traffic that stops turning into ordersSearches for milk are up. Orders for milk are down. What happened?Read the use case
- On-time delivery by stepDeliveries late at one step of fulfillmentThe delivery promise is 10 minutes. Which of the five steps just broke it?Read the use case
- Approval rateApproval rates shifting after a changeYou shipped a new model on Tuesday. Approvals dropped on Wednesday. Coincidence?Read the use case
- Settlement and interchange rate by merchantSettlement rates spiking for some merchantsOne missing tax field. Thousands of transactions downgraded. Who noticed?Read the use case
- Booking success by airlineBookings with one airline failing more oftenBookings for one airline just fell well below normal. Is it the airline, the channel or the region?Read the use case
From a warehouse connection to a working agent.
Dev Kathuria, Product Manager at Bicycle, connects BigQuery, describes the business focus and builds an agent, start to finish. Your team reviews what Bicycle inferred before anything goes live, and when a key metric moves, Bicycle goes from the top-level signal into the segment underneath it.
Walkthrough · 2 min 22 s · captions onBhaskar on the podcasts this season.
- A.I. RebelsThe LLM reads the business; machine learning does the math.Listen to A.I. Rebels (opens in a new tab)
- FINTECHTALKYou can't replace an analyst; you can give them a power tool.Watch FINTECHTALK (opens in a new tab)
- The Silicon Valley PodcastAnchor on the KPI instead of handing people 20 dashboards.Listen to The Silicon Valley Podcast (opens in a new tab)
- The Ignite PodcastTalking to your data has a place; running the business on it doesn't.Listen to The Ignite Podcast (opens in a new tab)
- Sources
- Anthropic, How Anthropic enables self-service data analytics with Claude, 3 June 2026.
- Painter, Chalamalasetti, Kassekert and Bate, Automating Pharmacovigilance Evidence Generation: Using Large Language Models to Produce Context-Aware SQL, JAMIA Open, 2024.
- Bicycle, Revenue Leaks: How eCommerce and Retail Companies Tackle the Silent Margin Killer, 2026 benchmark.
The research cited describes other organizations' systems and does not endorse Bicycle.
- About Bicycle
Bicycle helps high-velocity businesses see what changed, understand why, and act before the opportunity is gone. The KPI moves. Bicycle detects it, explains the business and technical causes, and recommends the next action. Data & Analytics governs every definition, evidence trail, and action rule underneath.
- More editions
Previous: Edition 01, September 2026. The dashboard is green. Revenue is still slipping.
See it on your own KPI
Try the live demo, no signup, or open your own account.