India’s Leading Domain & Web Hosting Provider
Support
Domain & Login
Blog
Microsoft Foundry & Azure OpenAI Enterprise AI Services
Microsoft Foundry is Microsoft's platform for building, evaluating and running generative AI applications and agents on Azure, offering Azure OpenAI and other models, Foundry Agent Service, and integration with Azure AI Search, Document Intelligence and Content Safety. NTSPL builds grounded assistants and agents on Foundry for Indian enterprises that need answers traceable to their own documents, with evaluation and guardrails.
Microsoft Azure services in scope
The Problem Microsoft Foundry (Foundry Models, including Azure OpenAI) and Foundry Agent Service Solve
Enterprise GenAI pilots in India usually stall at the same point. The demo answers well on ten curated questions, then real users ask about the clause in the 2019 amendment, the leave policy for contract staff in Gujarat, or a customer email written half in Hindi, and the model answers confidently and wrongly. The root causes are retrieval quality, not model choice: scanned PDFs never OCR'd, policies chunked mid-table, no metadata to filter by location or version, no access trimming, and no evaluation set to prove a change made things better rather than worse.
NTSPL treats an AI assistant as a retrieval and evaluation engineering problem. We extract documents with Azure AI Document Intelligence, chunk by structure, index in Azure AI Search with hybrid keyword and vector retrieval plus semantic ranking, and apply security trimming from Entra groups. Models are deployed in Microsoft Foundry with deployment types chosen for data residency and throughput. Every release is scored against a golden question set using Foundry evaluations for groundedness and relevance, and Azure AI Content Safety with Prompt Shields filters inputs and outputs.
When to Build on Microsoft Foundry and Azure OpenAI (and When Copilot or Classic ML Fits Better)
Signals that point to enterprise ai on Microsoft Azure
- Specialists spend hours a day searching contracts, policies or manuals to answer repeatable questions.
- A shared mailbox or contact centre receives more than 2,000 emails a day that are triaged manually.
- A GenAI pilot built on public chatbots was blocked by legal or InfoSec over data handling.
- You need answers with citations to source documents and access trimmed by user role.
- Content in English and Indian languages must be searched together, often from scanned documents.
- A regulator, board or auditor expects documented evaluation and guardrails before AI goes into production.
When a different approach is better
- If the need is personal productivity across Outlook, Word and Teams for knowledge workers, buy Microsoft 365 Copilot rather than building a custom assistant on Foundry.
- For structured predictions such as demand forecasting, credit scoring or predictive maintenance, a generative model is the wrong tool; use Azure Machine Learning with classical models trained on tabular data.
- When answers must be exact and deterministic, such as tax calculations or account balances, call the system of record through an API and show the result; do not let a language model compute or paraphrase it.
Enterprise AI Building Blocks: Foundry Models, Foundry Agent Service, Azure AI Search, Document Intelligence and Content Safety
| Service | Role in the architecture | Choose it when |
|---|---|---|
| Microsoft Foundry (Foundry Models, including Azure OpenAI) | Hosts GPT-family and other foundation models behind Azure endpoints with Standard, Global, Data Zone and Provisioned deployment types, plus model catalogue, evaluations and tracing in one project. | You need enterprise-grade access to frontier models with Entra authentication, private networking and Microsoft's data-handling commitments. |
| Foundry Agent Service | Runs agents that combine a model with tools such as Azure AI Search, function calling, code interpreter and connected APIs, managing threads, tool orchestration and tracing. | The assistant must act, not only answer: look up a ticket, draft a reply into a CRM, or route an email based on retrieved policy. |
| Azure AI Search | Indexes enterprise content with keyword, vector and hybrid retrieval, semantic ranker, integrated vectorization and document-level security filters, serving as the grounding layer for RAG. | Answers must come from your own documents with citations, filtered by user permissions, version or business unit. |
| Azure AI Document Intelligence | Extracts text, tables, key-value pairs and layout from scanned and digital PDFs and images using prebuilt layout, contract, invoice and custom models. | Source documents include scans, stamped agreements or complex tables that plain PDF text extraction mangles. |
| Azure AI Content Safety | Detects harmful content categories, jailbreak and indirect prompt-injection attempts through Prompt Shields, and checks groundedness of responses against sources. | The assistant faces employees or customers, ingests external email or documents, or operates in a regulated sector. |
| Microsoft Copilot Studio | Low-code tool to build agents published to Teams, websites and Microsoft 365 Copilot, with knowledge sources, topics and connectors, and the option to call Foundry-built agents. | The use case is an employee-facing Q&A or workflow assistant in Teams that business teams should maintain without developers. |
Grounded RAG and agent architecture on Microsoft Foundry
Grounded RAG and agent architecture on Microsoft Foundry
Users reach the assistant through Teams, a web app or an email pipeline, authenticated by Entra ID; an orchestration layer or Foundry agent screens the request with Content Safety Prompt Shields, retrieves permitted passages from Azure AI Search, and calls an Azure OpenAI model deployment to generate a cited answer or action. Documents flow from SharePoint and file stores through Document Intelligence into the index, and every interaction is traced and scored by Foundry evaluations.
Select any component
Grounded RAG and agent architecture on Microsoft Foundry: click any of its 8 components to see its role and the design choice NTSPL makes for it, or trace the 7-step primary path.
GenAI Engineering Decisions: Deployment Types, Chunking and Hybrid Retrieval, Agents and Evaluation Gates
Deployment type and data residency
Standard regional deployments process prompts in the chosen region, which matters for personal data under the DPDP Act and for RBI or SEBI-regulated entities; check which models are offered in Indian regions before committing. Global deployments give higher quotas and earlier model access but may process data in any Azure region. Use Provisioned throughput when steady volume is high enough that reserved capacity costs less than tokens and when you need predictable latency.
Chunking and indexing strategy
Chunk by document structure, not fixed character counts. For contracts, split at clause level and attach clause number, party and agreement date as metadata. For policies, split at heading with location and effective-date fields. Keep tables whole. Chunks of roughly 300 to 800 tokens with 10 to 15 percent overlap work for most corpora. Re-index on document change events rather than nightly full rebuilds.
Retrieval quality before model choice
Measure retrieval separately from generation. Hybrid search plus semantic ranker usually beats pure vector search by a clear margin on enterprise documents with codes, names and clause numbers. Use query rewriting with a small model to expand abbreviations and handle Hinglish. Only after retrieval recall on the golden set exceeds about 85 percent is it worth comparing generation models.
Agent versus single-shot RAG
Use plain RAG for question answering: fewer calls, lower latency and easier evaluation. Move to Foundry Agent Service when the task needs multiple tools or steps, such as classify an email, retrieve policy, look up the customer in the CRM and draft a response. Constrain agents with explicit tool schemas, a maximum step count, and human approval for any action that changes a system of record.
Evaluation gates and human review
Build a golden set with domain experts: real questions, expected answers and the source passages. Gate every prompt, index or model change on groundedness and relevance scores not regressing. In production, sample 2 to 5 percent of conversations weekly for expert review, track thumbs-down reasons, and add failures to the golden set. Without this loop, quality drifts silently as documents change.
How NTSPL Takes a Foundry RAG Assistant from Golden Questions to Production
Golden question set with domain experts
Collect 150 to 400 real questions with expected answers and source references from the people who answer them today, defining pass thresholds.
Document extraction and index build
Run Document Intelligence on the corpus, chunk by structure with metadata, and build an Azure AI Search index with hybrid retrieval and security filters.
Retrieval tuning
Measure retrieval recall on the golden set, tune chunking, semantic configuration and query rewriting until recall meets the threshold.
Model deployment and orchestration
Deploy generation and utility models in Foundry with the right deployment type, then build the RAG flow or agent with tool schemas and step limits.
Safety and evaluation gate
Configure Content Safety and Prompt Shields, run Foundry evaluations for groundedness and relevance, and block release if scores regress.
Pilot, feedback loop and scale-out
Launch to a pilot group, review sampled conversations weekly, feed failures back into the golden set, and expand by department.
What you receive
Securing Azure OpenAI Applications: Private Networking, Security Trimming, Prompt Shields and Data Handling
GenAI applications introduce threats traditional apps do not: prompt injection hidden inside retrieved emails or documents, over-broad retrieval that shows a user content they could not open directly, leakage of personal data into logs, and agents taking unintended actions. Controls focus on private endpoints and managed identities for Foundry and Search, security trimming at retrieval, Prompt Shields on prompts and documents, data minimization in logs, and human approval for agent actions.
- Private endpoints and disabled public network access on Foundry, Azure OpenAI and Azure AI Search resources
- Document-level security filters in Azure AI Search mapped to Entra group IDs
- Prompt Shields for direct jailbreaks and indirect prompt injection in retrieved content
- Managed identities between app, Foundry and Search instead of API keys
- Human-in-the-loop approval and step limits for agent tools that write to systems of record
Azure OpenAI Token Economics: Pay-as-You-Go, Provisioned Throughput, Search Units and Caching
Costs come from model tokens (input and output priced separately per model), or from Provisioned throughput units reserved per deployment; Azure AI Search billed by tier and search units with semantic ranker queries; Document Intelligence billed per page processed; and Content Safety per text record or image. For most assistants, output tokens and an oversized model choice dominate, followed by Search replicas sized for peak.
- Route classification and query rewriting to a small model, reserving the large model for final answers
- Use prompt caching and keep system prompts stable to benefit from cached-input pricing where supported
- Limit retrieved chunks to the top 3 to 6 after semantic ranking to cut input tokens
- Move steady high-volume workloads to Provisioned throughput once utilization justifies it
- Run Document Intelligence extraction once and re-index from stored output rather than re-processing pages
Microsoft Foundry vs Copilot Studio vs Microsoft 365 Copilot: which should we use for an enterprise assistant?
Decide on four criteria. Audience and channel: if the need is everyday productivity inside Outlook, Word and Teams for all knowledge workers, Microsoft 365 Copilot is the product to license. Ownership: if business teams should build and maintain a Teams-based Q&A or workflow agent over SharePoint and a few connectors, Copilot Studio is faster and needs little code. Control: if you need custom retrieval, specific chunking, evaluation gates, multiple models, high volume or a customer-facing channel, build on Microsoft Foundry with Azure AI Search. Cost: Microsoft 365 Copilot is per user, Copilot Studio is by capacity or message consumption, and Foundry is pay per token or provisioned. Many enterprises combine them, using Copilot Studio as the front end that calls a Foundry-built agent.
Key facts about enterprise ai on Microsoft Azure
Enterprise AI on Microsoft Azure in Practice
3 enterprise ai scenarios from Infrastructure (roads, metro and power transmission), Manufacturing conglomerate, Private sector bank: the starting position, the services chosen, the architecture, the rollout and the measured outcome.
Infrastructure company's legal team reviews contracts in hours, not days
HR policy assistant in Teams answers 15,000 employees in 4 languages
Private bank triages 12,000 customer emails a day with a Foundry agent
Enterprise AI on Microsoft Azure: Frequently Asked Questions
How is Azure OpenAI in Microsoft Foundry priced?
Pay-as-you-go deployments bill per million input and output tokens, with rates varying by model and deployment type. Provisioned throughput bills reserved capacity per hour or through monthly and annual reservations, independent of tokens. Azure AI Search, Document Intelligence and Content Safety are billed separately. NTSPL estimates monthly INR cost from expected questions per day, average retrieved context and answer length.
Does Microsoft use our prompts or documents to train models?
No. Microsoft states that prompts, completions, embeddings and training data submitted to Azure OpenAI are not used to train or improve the foundation models and are not shared with OpenAI. Data may be stored temporarily for abuse monitoring, and eligible customers can apply for modified abuse monitoring. Your indexed documents remain in your own Azure AI Search and storage resources.
Can prompts and data be processed within India?
Yes, if you use Standard regional deployments in an Indian region where the required model is offered, and keep Azure AI Search and storage in Central India or South India. Global deployments may process data outside India, so avoid them for personal data you have committed to keep in-country under the DPDP Act or for RBI, SEBI and IRDAI-regulated data. Model availability by region changes, so check before design.
We built a pilot on a public chatbot. How do we move it to Azure?
Start by turning the pilot's best questions into a golden evaluation set, then rebuild retrieval in Azure AI Search with your actual documents and access controls. Prompts usually transfer with modest changes. The main new work is extraction with Document Intelligence, security trimming, Content Safety configuration and evaluation gates, which typically take 6 to 10 weeks for one use case.
How do you stop the assistant from making things up?
By constraining it to retrieved sources and measuring the result. The prompt instructs the model to answer only from provided passages and to say when information is missing, citations are mandatory, and Foundry evaluations score groundedness on a golden set before each release. Content Safety groundedness detection can flag ungrounded responses at runtime. Hallucinations rarely drop to zero, so high-stakes uses keep a human reviewer.
Can it handle Hindi, regional languages and Hinglish?
Current GPT-family models understand and generate Hindi and major Indian languages well, and handle mixed Hinglish text. Retrieval is the harder part: use multilingual embeddings, and add query rewriting so that a Hinglish question matches English policy text. Document Intelligence OCR supports Devanagari and several other Indian scripts. NTSPL includes regional-language questions in the golden set to test this explicitly.