India’s Leading Domain & Web Hosting Provider
Support
Domain & Login
Blog
Generative AI on AWS: Amazon Bedrock, RAG and SageMaker AI Services
Generative AI and ML on AWS means building applications on Amazon Bedrock foundation models, grounding them in company documents with Knowledge Bases, constraining them with Guardrails, running agents on AgentCore, and training custom models in SageMaker AI. NTSPL builds these for Indian enterprises, starting with a measured evaluation set so answer accuracy, cost per query and data residency are known before rollout.
AWS services in scope
The Problem Amazon Bedrock and Amazon Bedrock Knowledge Bases Solve
Most enterprise GenAI pilots we review in India stall between demo and production for the same reasons. The assistant answers fluently but cites the wrong circular or an outdated manual, nobody measured accuracy against real questions, and the security team cannot say whether a junior employee can retrieve board papers by asking cleverly. Token costs that looked trivial in a pilot of 20 users balloon at 2,000. Regulated firms then ask where prompts are processed, and the answer involves cross-Region inference routes nobody documented.
NTSPL begins every engagement with an evaluation set of 150 to 400 real questions with expert-approved answers, because without it there is no way to choose a model, chunking strategy or prompt. We build retrieval on Bedrock Knowledge Bases with metadata filters tied to the user's entitlements, apply Guardrails with contextual grounding checks, and run agents on AgentCore with narrowly scoped tools. Each release is scored on accuracy, citation correctness, latency and cost per answer before users see it.
When Amazon Bedrock and SageMaker AI Are the Right Choice for Your AI Use Case (and When They Are Not)
Signals that point to generative ai & ml on AWS
- Employees spend more than 30 minutes a day searching manuals, circulars, research notes or past cases for answers.
- A GenAI pilot exists but has no measured accuracy, and stakeholders disagree on whether it is good enough.
- Expert drafting work, such as research summaries, regulatory sections or service reports, follows templates and draws on known sources.
- Customer or field queries are repetitive but need answers specific to product, model number or policy version.
- Security or compliance has blocked public chatbots and asked for an option where data stays in your AWS account.
- You have labelled historical data, such as tickets with failure codes, that could train a classifier to route or triage work.
When a different approach is better
- If the task is deterministic, such as calculating GST, applying a pricing rule or validating a form, an LLM adds cost and risk; write code or rules in Lambda and use GenAI only for the language around it.
- If you need a generic employee assistant over SharePoint and email with no custom workflows, building on Bedrock is slower than adopting Amazon Quick or a Microsoft 365 assistant you already license.
- If answers must be legally binding or individually suitable investment advice with no human review, do not automate them; keep a human approver and use GenAI for drafting and research only.
GenAI Building Blocks: Bedrock Models, Knowledge Bases, Guardrails, AgentCore and SageMaker AI
| Service | Role in the architecture | Choose it when |
|---|---|---|
| Amazon Bedrock | Provides API access to foundation models from Anthropic, Amazon Nova, Meta, Mistral, Cohere and others, with on-demand, batch and provisioned throughput options and no use of customer prompts to train base models. | You want production-grade LLM capability without hosting GPUs, and need to switch or compare models as the market moves. |
| Amazon Bedrock Knowledge Bases | Manages retrieval-augmented generation end to end: ingesting documents from S3 and other sources, chunking, embedding, storing vectors in OpenSearch Serverless, Aurora PostgreSQL or S3 Vectors, and returning cited passages, including natural-language queries over structured data. | Answers must come from your own policies, manuals, research or filings with citations, not from the model's general knowledge. |
| Amazon Bedrock Guardrails | Applies configurable content filters, denied topics, prompt attack detection, sensitive information redaction, contextual grounding checks and Automated Reasoning checks to model inputs and outputs. | You must block unsuitable advice, mask PAN or phone numbers, or reject answers not supported by retrieved sources. |
| Amazon Bedrock AgentCore | Runs agents built with any framework in isolated sessions, with Gateway to expose APIs and Lambda functions as tools, Identity for delegated user authorization, Memory for context and Observability for traces. | The assistant must take actions or call systems such as CRM, inventory or ticketing on the user's behalf, under that user's permissions. |
| Amazon SageMaker AI | Trains, fine-tunes, evaluates and hosts custom models on managed infrastructure, including smaller classifiers and fine-tuned open-weight models on dedicated endpoints. | You need a model trained on your own labelled data, such as failure-code classification or demand forecasting, or full control over model weights and hosting. |
| Amazon Quick (successor to Amazon Q Business) | Offers a managed workplace assistant with built-in connectors to enterprise sources such as SharePoint, Confluence and S3; Amazon Q Business is no longer open to new customers, and AWS directs new deployments to Amazon Quick. | You need a general internal Q&A assistant quickly with minimal custom code, and do not need custom workflows or domain-specific evaluation. |
Identity-aware RAG assistant with Bedrock Knowledge Bases, Guardrails and AgentCore tools
Identity-aware RAG assistant with Bedrock Knowledge Bases, Guardrails and AgentCore tools
A user's question reaches an API on API Gateway, which passes the user's identity and entitlements to an agent on AgentCore Runtime. The agent retrieves passages from Bedrock Knowledge Bases with metadata filters for that user, calls approved tools through AgentCore Gateway, and returns a cited answer only after Guardrails input and output checks pass.
Select any component
Identity-aware RAG assistant with Bedrock Knowledge Bases, Guardrails and AgentCore tools: click any of its 8 components to see its role and the design choice NTSPL makes for it, or trace the 7-step primary path.
GenAI Engineering Decisions: Model Choice, Retrieval Design, Grounding Checks and Agent Boundaries
Model selection by measured accuracy and cost per answer
We shortlist two or three Bedrock models, typically one frontier model and one smaller, cheaper model, and run the golden question set through each with identical retrieval. Smaller models often score within 2 to 5 points on straightforward lookup questions at a fraction of the token cost, while complex reasoning or long-document synthesis justifies the larger model. Routing simple questions to the small model and escalating hard ones is usually the best cost-quality trade.
Chunking, metadata and retrieval filters
Fixed 300-token chunks break tables and numbered clauses. For manuals and policies we use hierarchical chunking so retrieval returns the parent section, and semantic chunking for narrative research. Every chunk carries metadata: document type, product or model number, version, effective date and access group. Filtering by version and effective date eliminates the most common failure we see, the assistant quoting a superseded circular or service bulletin.
Grounding and refusal behaviour
Enable Guardrails contextual grounding checks with thresholds tuned on the evaluation set, typically 0.6 to 0.8 for grounding and relevance, so answers not supported by retrieved text are blocked or replaced with 'I could not find this in the approved sources'. A good assistant refuses 5 to 10 percent of questions. Zero refusals usually means it is inventing answers to fill gaps.
Agent tool boundaries
Each tool exposed through AgentCore Gateway gets an explicit schema, read or write classification and per-user authorization through AgentCore Identity. Write actions such as creating a service ticket or ordering a spare part require user confirmation in the interface. We never give an agent a database connection or a generic HTTP tool; prompt injection in a retrieved document must not be able to trigger an unreviewed action.
Inference location and data residency
Bedrock runs in ap-south-1, but not every model is offered there on-demand, and some are reachable only through cross-Region inference profiles that may process requests in other Asia Pacific Regions. Data at rest such as knowledge bases and logs stays in Mumbai. For workloads where regulators or contracts require in-country processing, we restrict to models available in-Region, or host an open-weight model on SageMaker AI in Mumbai.
How NTSPL Takes a Bedrock RAG Assistant from Evaluation Set to Production
Golden question set
Collect 150 to 400 real user questions with expert-approved answers and source references, split into lookup, synthesis and should-refuse categories.
Source curation and metadata
Select authoritative documents, remove superseded versions, and attach metadata for version, effective date, product and access group before ingestion into S3.
Knowledge Base and retrieval tuning
Create the Knowledge Base, test two chunking strategies and an optional reranker, and measure retrieval hit rate on the golden set before touching prompts.
Model, prompt and Guardrails evaluation
Score candidate models and prompts on accuracy, citation correctness, refusal rate, latency and cost per answer, then tune Guardrails thresholds.
Agent tools and identity
Deploy the agent on AgentCore Runtime, register read and confirmed-write tools through Gateway, and wire user identity end to end.
Pilot, feedback loop and regression gate
Release to 30 to 50 users, review thumbs-down answers weekly, add them to the golden set and block releases that reduce the score.
What you receive
Securing LLM Applications: Prompt Injection, Data Leakage, Guardrails and Identity-Aware Retrieval
LLM applications introduce attack paths that traditional apps do not: prompt injection hidden in uploaded or retrieved documents, users coaxing the model to reveal documents they are not entitled to, sensitive data echoed into answers or logs, and agents performing actions an attacker wrote into a PDF. Security must sit outside the model, in retrieval filters, tool authorization and output checks, because the model itself cannot be trusted to enforce access.
- Retrieval metadata filters built from the user's entitlements, so restricted chunks are never sent to the model
- Guardrails prompt attack filters and denied topics on input, and grounding plus PII redaction on output
- AgentCore Identity so every tool call carries the end user's permissions, not a service superuser
- Bedrock and AgentCore accessed through VPC interface endpoints with KMS-encrypted invocation logs
- Confirmation step in the UI for any write action an agent proposes
Bedrock Pricing: Input and Output Tokens, Prompt Caching, Batch Inference and Vector Store Costs
Bedrock on-demand pricing is per input and output token, and differs widely by model; output tokens cost several times more than input tokens. RAG answers carry retrieved passages as input, so a 5,000-token context per question dominates cost. Knowledge Bases add embedding and vector store costs, Guardrails bill per text unit assessed, and AgentCore bills for runtime and gateway usage. Latency is driven by context length and model size.
- Route simple lookups to a smaller model and escalate only complex questions to a frontier model
- Use prompt caching on supported models for long, repeated system prompts and instructions
- Use Bedrock batch inference, priced below on-demand, for overnight bulk summarization or drafting
- Limit retrieved passages to the top 5 to 8 after reranking instead of stuffing 20 chunks
- Pick S3 Vectors for large, infrequently queried indexes instead of always-on vector clusters
Amazon Bedrock vs SageMaker AI vs Amazon Quick: which should we use for enterprise GenAI?
Choose by how much you need to build and control. Amazon Quick, the successor to Amazon Q Business, fits a general internal assistant over common enterprise sources, live in weeks with little code, but with limited control over retrieval logic, evaluation and custom workflows. Amazon Bedrock fits custom applications: domain assistants, drafting tools and agents where you control prompts, retrieval, Guardrails and tools, paying per token with no infrastructure. SageMaker AI fits when you need to train or fine-tune your own models, host open-weight models on dedicated instances for residency or cost at very high volume, or build classical ML such as classifiers and forecasts. Most NTSPL deployments combine Bedrock for the language layer with SageMaker AI for a specialised model behind one of the agent's tools.
Key facts about generative ai & ml on AWS
Generative AI & ML on AWS in Practice
3 generative ai & ml scenarios from Wealth management, Industrial machinery, Pharmaceuticals (R&D and regulatory affairs): the starting position, the services chosen, the architecture, the rollout and the measured outcome.
Wealth firm's research assistant cuts RM client-prep time from 50 to 12 minutes
Machinery maker's field technicians resolve faults 30% faster with a maintenance assistant
Pharma R&D team drafts CTD quality sections in days with cited, reviewable AI drafts
Quick-commerce grocer cuts dark-store stock-outs with hourly SKU demand analytics
Generative AI & ML on AWS: Frequently Asked Questions
How much does a Bedrock-based enterprise assistant cost to run?
For 500 users asking about 10 questions a day with RAG context of about 5,000 tokens, model costs typically range from INR 1 to 6 lakh per month depending on model choice and routing, plus INR 30,000 to 1.5 lakh for the vector store, Guardrails and AgentCore. We publish a cost-per-answer figure during evaluation so the business case is based on measured usage.
Does Amazon Bedrock use our data to train models?
No. Amazon Bedrock does not use your prompts and completions to train AWS or third-party base models, and does not share them with model providers. Your documents, knowledge base vectors and invocation logs stay in your AWS account, encrypted with your KMS keys. Fine-tuned custom models are private to your account.
Can prompts and documents stay inside India?
Documents, knowledge bases, vector stores and logs can all stay in ap-south-1. Inference location depends on the model: some run on-demand in Mumbai, while others use cross-Region inference profiles that may process requests in other Asia Pacific Regions. For SEBI, RBI or IRDAI regulated workloads with strict residency, we select in-Region models or host an open-weight model on SageMaker AI in Mumbai.
We already ran a GenAI pilot on another platform. Can we move it to Bedrock?
Usually in four to six weeks. Prompts and orchestration code port with modest changes, documents are re-ingested into a Bedrock Knowledge Base, and existing embeddings are rebuilt because embedding models differ. The most valuable thing to carry over is your test questions; if the pilot had none, we build the golden set first and score both versions.
How do Bedrock Guardrails and Knowledge Bases stop the assistant from making things up?
By making it cite and refuse. Answers must quote retrieved passages, Guardrails contextual grounding checks block responses not supported by those passages, and the prompt instructs the model to say when sources do not cover the question. We measure hallucination rate on the golden set before release and track thumbs-down answers weekly after launch.
When should we fine-tune a model instead of using RAG?
Rarely for knowledge, often for format or classification. RAG is better for facts that change, because updating a document is instant while retraining is not. Fine-tuning on SageMaker AI or Bedrock helps when you need a consistent output style, domain terminology or a high-volume classification task where a small tuned model is cheaper and faster than prompting a large one.