India’s Leading Domain & Web Hosting Provider
Support
Domain & Login
Blog
Legacy Data Warehouse and Hadoop Migration to Cloud Analytics
Data modernization and migration moves a legacy analytical platform, such as Teradata, Netezza, an on-prem Oracle or SQL Server warehouse or a Hadoop cluster, onto a cloud warehouse or lakehouse, converting schemas, SQL, ETL jobs and reports. NTSPL runs these migrations with automated code translation, workload-by-workload waves and parallel runs that reconcile every certified output before cut-over.
Delivered on
In scope
Why Legacy Warehouse Exits Overrun
Legacy warehouses rarely get retired on time. An appliance support contract ends, the Hadoop distribution goes out of support, or a hardware refresh quote arrives at INR 8-15 crore, and the team discovers 6,000 stored procedures, 1,400 Informatica or SSIS jobs and 900 reports, a third of which nobody uses. Lift-and-shift of that sprawl to the cloud just moves the cost; rewriting it all by hand takes years. Neither option gets the old platform switched off.
NTSPL starts with a usage-based inventory to cut what will not move, typically 25-40% of objects. Remaining SQL and ETL are translated with automated tools, then corrected and tested by engineers. Migration proceeds by business workload, each running in parallel on old and new platforms until row counts, aggregates and report outputs reconcile, before the legacy job is switched off.
Triggers for Moving Off Appliances and Hadoop
Signals you need Data Modernization & Migration
- A Teradata, Netezza or Exadata support renewal or hardware refresh is due within 12-18 months
- A Cloudera or Hortonworks Hadoop cluster runs an out-of-support release or needs new nodes
- Nightly batch on the legacy warehouse exceeds its window more than once a week
- Licence and support for the legacy platform exceed INR 1 crore a year
When to choose something else
- You are moving an operational database (OLTP) to the cloud, with or without an engine change: see Database Migration or Managed Database Modernization.
- You have not decided the target pattern and operating model: run Data Strategy & Architecture first, or migration will recreate the old design.
Inventory, Conversion and Reconciliation Workstreams
Usage-based inventory
Query logs, job schedules and report usage analysed to classify every table, procedure, job and report as migrate, consolidate or retire.
Schema and SQL conversion
Automated translation of DDL, views, procedures and BTEQ, NZSQL, PL/SQL or HiveQL, with engineer correction of the remainder.
ETL and orchestration conversion
Informatica, SSIS, DataStage or Oozie jobs converted to dbt, Dataform, Spark or native pipelines with Airflow-style orchestration.
Historical data transfer
Bulk extract and transfer of history, online or by offline appliance where volumes and bandwidth require, with checksums.
Parallel run and reconciliation
Old and new platforms run side by side, with automated comparison of counts, sums and report outputs before cut-over.
How Data Modernization & Migration Works, Step by Step
Usage-Led Warehouse Migration Factory
Usage logs drive the inventory and the retire list, automated translators convert code, and history is transferred in bulk. Each workload then runs in parallel until reconciliation passes, after which the legacy job is decommissioned.
Select any component
Usage-Led Warehouse Migration Factory: click any of its 8 components to see its role and the design choice NTSPL makes for it, or trace the 7-step primary path.
Migration Tooling by Target Cloud
SCT converts Teradata, Netezza and Oracle DW schemas and code to Redshift; Hadoop Hive and Spark workloads move to EMR with Iceberg on S3.
SQL Server and Synapse estates move naturally to Fabric Warehouse; Hadoop Spark and Hive workloads usually land on Azure Databricks with Delta.
BigQuery Migration Service offers batch and interactive SQL translation from Teradata, Netezza, Oracle, Redshift and Hive dialects, plus migration assessment from query logs.
Target, Conversion and Cut-Over Choices
Like-for-like or redesign
Move core schemas largely like-for-like so reports can be reconciled, and redesign only where the cloud platform rewards it, such as partitioning and clustering, replacing summary tables with materialised views, or moving logs to the lakehouse. Full remodelling during migration doubles risk; schedule it as a later wave once parity is proven.
Convert ETL or rebuild as ELT
Mechanical conversion of Informatica or SSIS jobs to another GUI tool preserves complexity and licence cost. We rebuild as SQL-based ELT in dbt or Dataform where logic is mostly joins and aggregations, typically 70-80% of jobs, and use Spark only for heavy procedural or file processing. Keep conversion inventories per job for traceability.
Big bang or wave cut-over
Use waves by business workload, such as finance, sales, then supply chain, each with its own parallel run. Big bang only suits small estates under about 200 objects. Every wave must decommission its legacy jobs; otherwise you pay for two platforms indefinitely. Set a hard dual-run end date, typically 6-10 weeks per wave.
History transfer method
Online transfer works up to tens of terabytes over a 1 Gbps link in reasonable time. Above about 100 TB or with constrained links, use Azure Data Box or Google Transfer Appliance, or on AWS DataSync over a temporary Direct Connect link (Snowball Edge is closed to new customers), for the bulk and sync deltas online. Always transfer in open formats like Parquet with checksums, never as proprietary dump files.
Wave-by-Wave Migration Programme
Usage assessment and wave plan
Analyse logs, build the migrate-consolidate-retire register and agree workload waves.
Target foundation and translation set-up
Build the target platform, configure translators and convert a pilot workload.
History transfer
Bulk-move history in Parquet with checksums and set up incremental sync.
Wave conversion and parallel runs
Convert, test and dual-run each wave until two cycles reconcile cleanly.
Cut-over and decommission
Switch consumers, disable legacy jobs and terminate licences and hardware.
What you receive
Migration Progress and Parity Measures
| KPI | Typical target |
|---|---|
| Objects retired instead of migrated | 25-40% of inventory |
| Automated SQL conversion rate | Typically 70-90% depending on dialect |
| Reconciliation pass rate before cut-over | 100% of certified outputs |
| Dual-run period per wave | Under 10 weeks |
Migration Risks and How We Contain Them
| Risk | How NTSPL prevents it |
|---|---|
| Hidden consumers break when legacy tables disappear | Query logs identify every reader; unknown connections are traced before decommissioning. |
| Semantic differences, such as null handling, collation or rounding, shift report totals | Hash and aggregate reconciliation per table catches drifts; dialect quirks are logged in a conversion rulebook. |
| Dual running drags on and doubles cost | Each wave has a fixed end date and steering committee review if it slips. |
Lift-and-shift vs re-platforming a legacy data warehouse to the cloud: which is safer?
Re-platforming to a cloud-native warehouse or lakehouse, done in waves with parallel runs, is safer over the life of the platform. Lifting a Teradata or Hadoop stack onto cloud VMs keeps the licences, the administration effort and the batch-window problems, and often costs more than on-premises. The real safety comes from reconciliation, not from avoiding change: if every certified output matches for two cycles before cut-over, the business sees no difference on day one. Lift-and-shift only makes sense as a short bridge when a data centre exit date cannot move.
Key facts: Data Modernization & Migration
Data Modernization & Migration in Practice
2 Data Modernization & Migration scenarios, from Cement and Airline (domestic carrier), plus related platform case studies.
Cement group exits an Oracle warehouse and OBIEE, cutting run cost 46%
Airline moves an out-of-support Hadoop cluster to an Iceberg lakehouse on AWS
EPC company vacates its owned data centre in 7 months and halves run cost
Agri-inputs company sees dealer secondary sales daily instead of 45 days late
Data Modernization & Migration: Frequently Asked Questions
How long does a Teradata or Hadoop migration take?
A mid-size estate of 20-80 TB with 2,000-5,000 objects typically takes 6-9 months in three to five waves. Costs commonly range from INR 80 lakh to 3 crore depending on code volume and the share of logic that cannot be auto-converted.
How much SQL can be converted automatically?
Tools such as BigQuery Migration Service and AWS SCT usually convert most standard SQL and DDL. Procedural code, vendor-specific functions and dynamic SQL need engineer work. We measure the rate on a pilot workload in the first month rather than promise a figure upfront.
Can we keep historical data in India during migration?
Yes. Where an offline appliance is offered for India it is loaded within India; otherwise bulk data moves online over private links. Either way data is ingested into Indian regions such as AWS Mumbai, Azure Central India or Google Cloud Delhi. Chain-of-custody and encryption keys remain under your control throughout. Transfer logs are kept as audit evidence.
What happens to our existing BI reports?
They are re-pointed to the new platform wave by wave and compared output by output with the legacy versions. Reports unused for 90 days or more are retired rather than migrated, after owners confirm. Users approve each report before the switch.
Will the cloud platform be cheaper than our appliance?
Usually, once dual running ends and licences are terminated, often by 30-50% on total run cost. Savings depend on tuning partitioning and compute; a lift of the old design without tuning can cost more, so FinOps controls are built in from the first wave.
Do we need to freeze changes on the legacy warehouse?
Only for objects in the wave being cut over, typically for two to four weeks. Other areas continue normally, and any changes made during a wave are tracked and applied to both platforms until cut-over. Freeze windows are agreed per wave.