India’s Leading Domain & Web Hosting Provider
Support
Domain & Login
Blog
Gemini & Vertex AI Development: Agents, RAG and Custom ML Models
Generative AI and ML on Google Cloud means building with Gemini models and custom models on the Gemini Enterprise Agent Platform, the April 2026 evolution of Vertex AI, grounded in your own data through Agent Search. NTSPL designs, evaluates and runs multilingual agents, retrieval systems and vision models for Indian manufacturers, consumer brands and public services, with measurable accuracy gates before launch.
Google Cloud services in scope
The Problem Gemini models (Pro, Flash and Flash-Lite tiers) and Gemini Enterprise Agent Platform (formerly Vertex AI) Solve
Generative AI pilots in India rarely fail on model quality. They fail because the assistant quotes a warranty term from a superseded PDF, answers in formal Hindi when the customer typed Hinglish, invents a bus fare, or costs four times the forecast. Vision projects fail differently: a defect model that scored 98 percent on curated images misses scratches under a new shift's lighting. The engineering problem is grounding, evaluation and serving cost: getting answers tied to authoritative sources, measuring them against an agreed test set in each language, and choosing models and throughput so the unit cost per conversation or per inspected part makes business sense.
NTSPL builds evaluation first. Before a prompt is written we assemble a golden set of 300 to 1,000 real questions or images per use case, labelled by the business, including Indian-language and code-mixed inputs. Agents are built with the Agent Development Kit on Google Cloud's Agent Platform, grounded through Agent Search over curated content and tool calls to live systems, and protected by Model Armor. Every release must beat the previous version on groundedness, task success and cost per interaction before it ships.
When Gemini Agents, Agent Search or Custom Models Are the Right Fit (and When They Are Not)
Signals that point to generative ai & ml on Google Cloud
- Support or helpdesk volume exceeds what agents can handle, and 50 percent or more of queries are answerable from manuals, policies or status systems.
- Customers write in Indian languages or code-mixed Hinglish, Tanglish or Benglish and your current bot handles only English keywords.
- Manual visual inspection misses defects at a measurable rate, or inspectors cannot keep pace with line speed.
- A GenAI pilot exists but nobody can state its accuracy, hallucination rate or cost per conversation.
- Documents such as invoices, claims or delivery notes are keyed in manually at hundreds per day.
- Leadership asks for AI use cases with a defensible business case and governance before budget approval.
When a different approach is better
- If the answers are a fixed decision tree with fewer than 50 intents and no free text, a rules-based flow in Conversational Agents or your existing IVR is cheaper and more predictable than a generative agent.
- If you need forecasting or propensity scores on tabular warehouse data, BigQuery ML is faster to deliver than a custom Agent Platform model.
- If employees simply need search and chat over company documents, buy Gemini Enterprise rather than building a custom agent.
Google Cloud AI Building Blocks: Gemini Models, Agent Platform, Agent Search, Model Garden and Gemini Enterprise
| Service | Role in the architecture | Choose it when |
|---|---|---|
| Gemini models (Pro, Flash and Flash-Lite tiers) | Multimodal foundation models that read text, images, audio, video and documents, call tools and produce structured output, served through the Agent Platform API with regional endpoints. | You need reasoning over unstructured content or conversation in English, Hindi and other Indian languages, with images or documents as input. |
| Gemini Enterprise Agent Platform (formerly Vertex AI) | Google Cloud's managed AI platform for building, evaluating, deploying and governing agents and models, including the Agent Development Kit, managed agent deployment, custom training, pipelines, model registry and endpoints. | AI moves from a notebook demo to a production service that needs versioning, evaluation, monitoring and IAM-controlled deployment. |
| Agent Search (formerly Vertex AI Search) | Managed retrieval over websites, PDFs, structured data and connected sources with parsing, chunking, ranking and grounded answer generation with citations. | Answers must come from your manuals, policies, timetables or knowledge base, and building a custom vector pipeline would slow delivery. |
| Model Garden | Catalogue of Google, partner and open models, such as Gemma and third-party models, that can be called as managed APIs or deployed to your own endpoints. | A use case needs an open model you can fine-tune and host, or a comparison across model families before committing. |
| Model Armor | Screens prompts and responses for prompt injection, jailbreak attempts, harmful content and sensitive data, and can be applied in front of agents and model endpoints. | The assistant faces the public or reads untrusted documents and you must block instruction injection and data leakage. |
| Custom training and endpoints on Agent Platform | Runs training jobs on GPUs or TPUs for task-specific models such as defect detection, registers versions and serves them online or exports them for edge deployment. | The task needs millisecond inference on images or sensor data at volume, where a general multimodal model would be too slow or costly. |
| Gemini Enterprise | Employee-facing agent and search workspace that connects to company content such as Google Workspace, Microsoft 365, Confluence and Jira, with permissions-aware answers. | The goal is internal knowledge assistance for staff rather than a customer-facing product you build and brand yourself. |
Grounded multilingual agent with tool calls and evaluation gate
Grounded multilingual agent with tool calls and evaluation gate
A customer message from web, app, WhatsApp or voice reaches an agent built with the Agent Development Kit, passing first through Model Armor and, for voice, Speech-to-Text. The agent retrieves from Agent Search and calls live APIs through scoped tools, Gemini composes a cited answer, and every conversation is logged for evaluation against the golden set before each release.
Select any component
Grounded multilingual agent with tool calls and evaluation gate: click any of its 8 components to see its role and the design choice NTSPL makes for it, or trace the 6-step primary path.
Generative AI Design Decisions: Model Tiering, Grounding Strategy, Evaluation Gates and Serving Capacity
Model tiering and routing
We default to a Flash-tier Gemini model for conversation, retrieval synthesis and classification because it handles most support turns at a fraction of Pro-tier cost and latency. Pro-tier models are reserved for multi-step reasoning, long document comparison or tool plans with several calls, and are invoked by explicit routing rules rather than for every turn. Flash-Lite handles high-volume classification such as language and intent detection. We re-run the golden set on each new model version before switching.
Grounding: Agent Search, custom vector store or structured lookup
Agent Search is our default for document grounding because parsing, chunking, ranking and citations come managed. We build a custom pipeline with embeddings and a vector index in AlloyDB or BigQuery only when chunking must follow domain structure or retrieval must join with transactional filters. Numbers such as prices, fares and warranty periods always come from a structured tool call, never from retrieved prose, and the answer must cite the source used.
Indian-language and code-mixed handling
We test each target language separately, because accuracy in Hindi does not predict accuracy in Odia or Assamese. The golden set includes native-script, romanized and code-mixed inputs in realistic proportions from real logs. Responses mirror the user's script and register; a user typing romanized Hindi gets romanized Hindi. For voice, Chirp models in Speech-to-Text are evaluated on word error rate with real call audio, including background noise, before the agent itself is assessed.
Evaluation gates and release criteria
Every use case gets explicit thresholds agreed with the business, for example groundedness above 95 percent, task success above 85 percent, and harmful or off-policy answers at zero on the red-team set. We combine model-based scoring with human review of a stratified 10 percent sample per language. Prompt, retrieval or model changes go through the same pipeline as code. Production conversations with low ratings or handoffs feed back into the golden set weekly.
Serving capacity: pay-as-you-go, Provisioned Throughput or batch
Standard pay-as-you-go suits pilots and variable traffic. Once a use case sustains predictable peaks, such as an evening support rush, Provisioned Throughput reserves capacity and protects latency, sized from measured tokens per second at peak. Offline jobs such as summarizing a day's calls or re-tagging a catalogue use batch inference at lower cost. Context caching is applied when long system prompts or product manuals are reused across many turns.
How NTSPL Takes a Gemini Agent from Golden Test Set to Production
Use-case sizing and golden set
Pull 3 to 6 months of real tickets, calls or images, choose the target intents or defect classes, and label a golden set per language or product line.
Content curation and grounding
Clean and tag source documents, load Agent Search data stores, and expose structured facts through typed tool APIs.
Agent or model build
Implement the agent with the Agent Development Kit or train the custom model, iterating against the golden set rather than ad-hoc demos.
Safety and red-team
Apply Model Armor, test prompt injection, off-topic and abusive inputs in every supported language, and fix failures before pilot.
Shadow and pilot
Run in shadow mode beside human agents or inspectors, then pilot on 10 to 20 percent of traffic with handoff paths.
Scale and capacity
Move to full traffic, size Provisioned Throughput from measured peaks, and start the weekly failure review cycle.
What you receive
Securing Gemini Agents: Prompt Injection Defence, Tool Permissions and Personal Data in Prompts
Generative agents introduce new attack paths: prompt injection through user messages or retrieved documents, tools that perform actions on behalf of an attacker, and personal data flowing into prompts and logs. We treat the model as untrusted: it can only call narrowly scoped tools under its own service account, write actions need user confirmation, and Model Armor screens both directions. Google states it does not use customer prompts and outputs from Agent Platform to train its foundation models.
- Model Armor screening of prompts and responses for injection, jailbreaks and sensitive data
- Per-tool service accounts so a manipulated agent cannot reach unrelated APIs
- Redaction of phone numbers, Aadhaar and order identifiers before transcripts reach logs
- VPC Service Controls around Agent Platform, Agent Search and BigQuery projects
- Allowlisted data sources in Agent Search so untrusted web content cannot be retrieved
Gemini Cost Model: Token Pricing, Context Caching, Provisioned Throughput and Batch Inference
Gemini costs scale with input and output tokens per model tier, multiplied by turns per conversation, so long system prompts and retrieved context are the hidden driver. Agent Search adds query and storage charges, Speech-to-Text bills per audio minute, and custom vision models bill for training hours and endpoint node hours. We model cost per resolved conversation or per inspected part, not per million tokens.
- Flash-tier models by default, Pro-tier only through explicit routing for complex turns
- Context caching for long manuals or system prompts reused across many sessions
- Trimming retrieved chunks to the top 3 to 5 passages instead of stuffing full documents
- Provisioned Throughput only for sustained, predictable peaks; batch inference for offline jobs
- Edge deployment of vision models to avoid per-image cloud inference at line speed
Agent Search vs a custom RAG pipeline vs fine-tuning Gemini: how should we ground an assistant in our own data?
Decide on content type, freshness and control. Agent Search is the fastest route for documents, websites and FAQs: managed parsing, ranking and cited answers, with sync from Cloud Storage or connectors, and it suits most support and helpdesk cases. A custom RAG pipeline with your own embeddings and vector index fits when chunking must follow domain structure or retrieval must filter on live transactional data. Fine-tuning changes style, format or domain vocabulary, not knowledge; it does not keep answers current when policies change, so it complements retrieval rather than replacing it. For most Indian enterprises, start with Agent Search plus structured tool calls, measure groundedness, and add custom retrieval only for the queries that fail.
Key facts about generative ai & ml on Google Cloud
Generative AI & ML on Google Cloud in Practice
3 generative ai & ml scenarios from Consumer electronics, Automotive components manufacturing, Public sector, state road transport: the starting position, the services chosen, the architecture, the rollout and the measured outcome.
Electronics brand resolves 58 percent of support chats in six languages with Gemini
Auto-parts maker catches 94 percent of machining defects with an edge vision model
State bus operator answers commuter queries in four languages on voice and WhatsApp
D2C beauty brand reallocates 18 percent of ad spend after unifying attribution in BigQuery
Generative AI & ML on Google Cloud: Frequently Asked Questions
How is Gemini priced, and what does a support conversation cost?
Gemini is billed per input and output token, with rates depending on the model tier, plus charges for grounding services such as Agent Search. A typical six-turn support conversation on a Flash-tier model with retrieved context often costs a fraction of a rupee to a few rupees, depending on context size. NTSPL models cost per resolved conversation from your real transcripts before build, and uses caching and routing to keep it within the agreed target.
We already have a keyword chatbot. Do we throw it away?
Not necessarily. NTSPL often keeps deterministic flows for transactions such as payments or cancellations and places a Gemini agent in front for free-text understanding and knowledge questions. The existing bot's logs become the starting golden set. Over two or three releases, intents that the agent handles better are migrated, and the old flows remain for actions requiring strict control.
Can data stay in India when we use Gemini on Google Cloud?
Largely yes, with checks per model. Stored data such as Agent Search indexes, BigQuery tables and logs can be kept in asia-south1 or asia-south2, and NTSPL uses regional endpoints for model calls where the chosen model is served in an India region. Model availability by region changes, so we confirm it at design time and document any processing location for your DPDP Act records.
How well does Gemini handle Indian languages?
Well for major languages such as Hindi, Bengali, Tamil, Telugu, Marathi, Kannada, Gujarati and Malayalam, but quality varies by language and register. Code-mixed inputs and regional transliteration are where most errors appear. NTSPL measures accuracy separately per language on real user messages and sets launch thresholds per language, sometimes launching a few languages first.
What is the difference between Gemini Enterprise and building on Agent Platform?
Gemini Enterprise is a ready product for employees: search, chat and agents over company content with existing permissions. Agent Platform is the developer platform for building your own customer-facing agents, models and applications. Choose Gemini Enterprise for internal productivity, and Agent Platform when the assistant is part of your product, uses your brand, or needs custom tools and evaluation.
How long does a production Gemini use case take?
Typically 8 to 14 weeks for a first use case. About two weeks go to building the golden set and curating content, four to six weeks to building and evaluating, and two to four weeks to shadow and pilot. Vision inspection projects can take longer if new image capture hardware or lighting changes are needed at the line.