Conversational Analytics: Ask Business Questions, Get Governed Numbers
Conversational analytics lets business users ask questions in plain language, such as 'which distributors missed target in Q2', and receive answers computed from governed data through generated SQL or semantic-model queries. NTSPL builds these on top of existing warehouses and semantic layers for sales, operations and finance teams, with certified metrics, row-level security and measured answer accuracy.
Delivered on
In scope
From a Typed Question to a Trusted Number: How the Pipeline Works
Dashboards answer the questions someone anticipated last year. A regional sales manager who wants beat-wise numeric distribution for one SKU family in two districts waits three days for an analyst, or exports to Excel and builds a wrong pivot. Text-to-SQL demos look impressive on a clean sample schema and fall apart on a real warehouse with 900 tables, cryptic column names and three different definitions of net sales. Worse, a confidently wrong number in a chat window travels further than a wrong number in a report.
NTSPL grounds conversational analytics in a semantic layer, not raw tables. We start by certifying 40-120 business metrics and their dimensions with finance and business owners, expose them through Power BI semantic models, LookML or a curated view layer, and only then let a model translate questions. Every answer shows the metric definition, filters applied and the generated query. We measure execution accuracy on a 200-400 question test set built with the users, and route unsupported questions to an analyst queue.
When Business Users Need Ad-Hoc Answers Beyond Dashboards
Signals you need Conversational Analytics
- Analysts spend more than 40% of their week on one-off data requests from sales, operations or finance users.
- A governed warehouse or lakehouse already exists, but 70% of business users never open the BI tool.
- Field and regional managers need answers on mobile, often in Hinglish, between visits rather than at a desk.
- There are documented metric definitions, or leadership is willing to certify them, for the questions people ask most.
- Row-level access rules already exist in BI and must apply equally to any new question interface.
When to choose something else
- If there is no governed data model yet and metrics are computed differently in every report, start with data strategy and BI semantic modelling; chat on top of chaos produces faster chaos.
- If users mainly need the same 15 KPIs daily, a well-designed dashboard with alerts is cheaper and clearer than a chat interface.
- If the questions are about documents and policies rather than numbers, a RAG knowledge assistant is the right fit.
What We Build: Metric Layer, Question Engine and Answer Controls
Question inventory
Collect 200-400 real questions from analyst request queues and user interviews, tagged by metric, dimension, time grain and persona.
Metric certification
Agree definitions, formulas, grain and owners for 40-120 metrics, resolving conflicting versions such as gross, net and billed sales.
Semantic layer exposure
Publish certified metrics through Power BI semantic models, LookML explores or curated SQL views with business-friendly names and descriptions.
Question engine
Configure the platform data agent or a custom text-to-SQL service with schema descriptions, sample queries, glossary and few-shot examples.
Answer controls
Show definition, filters and query with each answer; clarify ambiguous terms; refuse unsupported questions with an analyst hand-off.
Evaluation and monitoring
Execution-accuracy test suite, weekly sample review of live answers and a feedback loop into glossary and examples.
Semantic-Layer-Grounded Natural Language Query Architecture
Semantic-Layer-Grounded Natural Language Query Pipeline
A user question is resolved against a business glossary and certified metric catalogue, translated into a semantic-model or SQL query, executed under the user's own row-level security and returned with its definition and query. Ambiguous or unsupported questions trigger clarification or an analyst hand-off.
Select any component
Semantic-Layer-Grounded Natural Language Query Pipeline: click any of its 9 components to see its role and the design choice NTSPL makes for it, or trace the 7-step primary path.
Fabric Data Agents, Looker Conversational Analytics and Amazon Quick Sight
Quick Sight topics provide governed natural-language querying over curated datasets; for complex schemas we build a Bedrock text-to-SQL service over Redshift or Athena with Lake Formation permissions applied per user.
Fabric data agents answer questions over semantic models, lakehouses and warehouses, and Copilot in Power BI works on the same models; good model descriptions and verified answers drive accuracy more than prompt tuning.
LookML provides the governed metric layer that Looker's conversational analytics queries; the Conversational Analytics API lets us embed the same capability in mobile apps or Google Chat with BigQuery as the engine.
Semantic Layer, Ambiguity Handling and Accuracy Design Choices
Semantic layer versus raw-schema text-to-SQL
Generating SQL against hundreds of raw tables typically lands at 50-70% execution accuracy on real enterprise questions. Grounding on a semantic layer with certified metrics and fixed joins routinely reaches 85-95%, because the model chooses metrics and filters rather than inventing joins. We use raw-schema generation only for analyst-facing tools where users can read and correct SQL.
Handling ambiguity
Terms like sales, last quarter or top customers are ambiguous in most companies. The engine should ask one clarifying question, or state its assumption explicitly in the answer, never silently choose. We maintain default interpretations agreed with owners, such as financial year April to March and net sales after returns, and display them on every answer.
Verified examples and descriptions
The biggest accuracy gains come from 30-80 verified question-and-query pairs and clear descriptions on every metric and column, not from model upgrades. We have analysts write descriptions in business language with units, grain and exclusions. Each new failed question in production becomes either a glossary fix or a new verified example.
Guardrails on execution
Generated queries run read-only, with row limits, a scanned-bytes or capacity cap and a 30-60 second timeout. Questions that would scan beyond the cap are refused with a suggestion to narrow the time range. Users never get access to data they could not see in their existing reports, because execution uses their own identity.
Metric Certification to Conversational Roll-out
Question harvest
Mine analyst request queues and interview 20-40 users to collect and tag 200-400 real questions.
Metric certification
Workshops with finance and business owners to agree definitions, grain and defaults for priority metrics.
Semantic layer hardening
Add or fix measures, relationships, descriptions and synonyms in the semantic model or LookML; verify row-level security.
Question engine configuration
Configure the data agent or custom service, add verified examples and build the execution-accuracy test suite.
Pilot with one persona
Release to one user group, such as regional sales managers, and review a daily sample of answers with analysts.
Scale and govern
Add personas and metrics in waves, with each wave gated on test-suite accuracy.
What you receive
Execution Accuracy and Adoption Measures for Data Chat
| KPI | Typical target |
|---|---|
| Execution accuracy on test questions | 85-95% for in-scope questions |
| Clarification or refusal on ambiguous or unsupported questions | Above 90% handled correctly |
| Analyst ad-hoc request volume | 30-50% reduction |
| Weekly active business users | 40-60% of target persona within 3 months |
| Median time to answer | Under 15 seconds |
Wrong Numbers, Leaked Rows and Other Conversational BI Pitfalls
| Risk | How NTSPL prevents it |
|---|---|
| Confidently wrong numbers erode trust | Answers show definition, filters and query; accuracy is gated by the test suite, and low-confidence questions are routed to analysts. |
| Users see rows outside their territory or entity | Execution under the user's identity so existing row-level security applies, verified by persona-based tests before launch. |
| Runaway query cost from broad questions | Scanned-bytes or capacity caps, row limits and timeouts, with refusals suggesting a narrower question. |
| Metric definitions drift as business rules change | Named metric owners, change history in the catalogue and automatic re-run of the test suite after any model change. |
Conversational analytics vs traditional BI dashboards: do we still need dashboards?
Yes. Dashboards remain the right tool for the 10-20 KPIs a role checks every day, for trend monitoring and for board packs, because they are consistent, scannable and cheap to serve. Conversational analytics covers the long tail: the one-off cut, the follow-up question and the field manager who will never open a BI tool. Both should sit on the same certified semantic layer so that a number in chat matches the number on the dashboard. Build or fix the semantic model first, keep dashboards for monitoring, and add conversation for exploration.
Key facts: Conversational Analytics
Conversational Analytics in Practice
2 Conversational Analytics scenarios, from FMCG (packaged foods sales) and Aviation (airline operations control), plus related platform case studies.
FMCG area sales managers get beat-level answers on mobile in 12 seconds, not 3 days
Airline operations control answers on-time performance questions in chat during the shift
Dairy cooperative sees milk procurement by collection centre within 15 minutes
D2C beauty brand reallocates 18 percent of ad spend after unifying attribution in BigQuery
Conversational Analytics: Frequently Asked Questions
How accurate are AI answers on business data?
On a well-described semantic layer with verified examples, 85-95% of in-scope questions return the correct number. Accuracy against raw warehouse tables is much lower. We measure execution accuracy on your own question set before launch and publish the result to users, along with the types of questions the tool will refuse.
What does conversational analytics cost to implement?
If a governed semantic model already exists, a first persona can go live in 8-12 weeks with indicative NTSPL fees of ₹12-28 lakh. Most of the effort is metric certification and descriptions. Platform cost depends on Fabric capacity, Looker licences or Quick Sight seats, plus modest model usage.
Will users see data they are not allowed to see?
No, if designed correctly. Queries run under each user's own identity, so the same row-level and column-level security as their existing reports applies. We test with accounts from each persona, such as a regional manager and a national head, to confirm results differ as expected.
Can users ask questions in Hindi or Hinglish?
Current models handle Hinglish questions about business data reasonably well, especially when the glossary includes common local terms. We include Hinglish questions in the test set and add synonyms for terms users actually type. Answers can be phrased back in the user's language while numbers come from the governed query.
Where is the data processed, and does DPDP apply?
Queries run in your existing warehouse region, such as Azure Central India, AWS Mumbai or Google Cloud Mumbai. Only schema context and the question go to the model; result rows may be summarised by it. Where results include personal data, the DPDP Act 2023 applies, so we restrict models to in-region deployments.
Which questions should not go to conversational analytics?
Questions requiring forecasts, statistical tests or data that is not in the governed layer should be routed to analysts. We configure the tool to recognise these and hand them off rather than improvise. Over time, frequent hand-offs show which metrics to add to the semantic layer next.
Scoping Conversational Analytics?
Start with the question buyers ask first: “Conversational analytics vs traditional BI dashboards: do we still need dashboards?” NTSPL will answer it for your estate and give you a fixed-scope proposal.