Leading AI models get fewer than four in ten business queries right without a semantic layer, exposing a critical barrier to AI moving into financial, operational, and other business-critical workflows. Every model improved, and an open-weight model kept pace with frontier models.

AtScale, a leading provider of the universal semantic layer for enterprise AI and analytics, today released research showing that adding its semantic layer doubled AI accuracy on business questions on average 1.7x to 2.4x with it, with gains on all seven AI models tested. The findings demonstrate how shared business definitions and governed computation can help enterprises improve AI reliability as they expand adoption across finance, sales, and operations.

As businesses race to adopt the latest AI models, many overlook a more fundamental question: does the AI understand what their business data actually means? Without that understanding, even the strongest model has to guess at definitions like revenue, margin, and active customer. In AtScale's research, leading models answered fewer than four in ten business questions correctly when they queried the database directly, even with the schema, column documentation and business glossary provided. That gap is what keeps AI out of the financial and operational workflows where the numbers have to be right.

AtScale conducted the research using BIRD-Interact, an independently developed benchmark with 600 tasks across 22 databases. 410 of the tasks are business questions, and 317 of those have a verifiable correct answer. Its ambiguous questions, purposefully messy data, and simulated-user conversations reflect challenges encountered in enterprise AI workflows.

The principal findings:

Every model improved. On the 410 business questions, the AtScale semantic layer doubled accuracy on average across all seven models tested (1.7x to 2.4x).
Open-weight models rival frontier models. With the AtScale semantic layer, Kimi K2.5 scored within 1.2 points of OpenAI's GPT-5 business questions.
The model that asks the most questions gained the most on follow-ups. Google's Gemini 3.1 Pro asked more clarifying questions than any other model tested. Without a semantic layer, it passed just 10.2% of follow-up questions, the lowest of any model. With AtScale, it led every model at 45.6%, a 345% lift.
Accuracy doubled on questions with a correct answer. On the 317 business questions with a verifiable answer, Claude Sonnet 5's accuracy rose from 46% to 93%.
The benchmark reveals a path to 100% accuracy. BIRD-Interact tests the four places where AI guesses on real business questions. The results point to a framework that addresses each one: disambiguating prompts, resolving vague semantics, handling dirty data, and calculating complex multi-pass metrics.

Jay Schuren, President, AtScale, added: “Large language models are probabilistic by design. Finance and operations demand consistent, auditable answers. That mismatch is one of the biggest barriers to putting enterprise AI into production. A semantic layer bridges the gap: the model reasons about the question, while governed business definitions and deterministic computation produce the answer. That’s how AI agents move from experimentation into the workflows that run the business, and how model providers unlock the next wave of enterprise adoption.”

For customers, the value extends beyond a benchmark score: a shared definition of revenue across finance and sales, consistent calculations across agents and dashboards over time, and the freedom to adopt new AI models without rebuilding business logic. AtScale gives enterprises a governed foundation for moving AI into everyday decisions, with a repeatable way to measure and improve accuracy as their business evolves.

The research supports AtScale for AI, announced separately today, which brings together semantic modeling, question clarification, governed computation, and accuracy evaluation. Powered by AtScale’s AI Computation Engine, the suite helps enterprises extend trusted business metrics into AI workflows and continuously evaluate the answers those workflows produce.

The preliminary, AtScale-reported findings use the best of three runs per model with the AtScale semantic layer, following the BIRD-Interact leaderboard convention, compared with one run per model without it. Results on the 317 answerable questions apply the same grading corrections to both approaches, compare against the average of three runs without the semantic layer, and are not comparable to the leaderboard's success rate. The full methodology is available in the Benchmark eBook.

AtScale will be hosting a webinar “Make Your AI Show Its Homework” about the Bird-Interact Benchmark results on October 28, 2026 at 2PM ET. Register here to learn more.

About AtScale

AtScale helps enterprises turn data into trusted answers for AI and analytics through a universal semantic layer that delivers consistent business metrics across models, agents, applications, and dashboards. Learn more at www.atscale.com.

View source version on businesswire.com: https://www.businesswire.com/news/home/20261015215972/en/

Media: Nicole Francoeur, nicole.francoeur@atscale.com

Logo

Tags

More Like This

AtScale Launches AtScale for AI to Close the Enterprise AI Accuracy Gap

AtScale Named an OpenAI Select Partner

Gen-AI tools deliver unprecedented, near-perfect data accuracy

ThoughtSpot Introduces Spotter Semantics to Bring Trust and Context to Enterprise AI

Sigma Launches Native Semantic Layer Integration and AI SQL Capabilities on Snowflake AI Data Cloud

GoodData Ushers in Era of Governed and Trustworthy AI Analytics

Actian Unveils Conversational Analytics Solution with Intelligently Generated Semantic Foundation for Trusted Insights

GoodData Launches GoodData AI: Embed, Extend, and Scale Analytics with AI