Skip to content
METRIC OAK

New: the 3-week Data Platform Assessment: fixed fee, priced roadmap. See what's included →

Migration

How long does a data warehouse migration take?

What actually drives the timeline of a migration to Databricks, Snowflake or Fabric: object counts, procedures, downstream consumers and reconciliation, the phases of a typical plan, and how to estimate your own.

Hemant Walchale

Hemant Walchale
· 3 min read

“It’s just copying data, why will it take until June?” is one of the most common questions in data platform migrations. The honest answer: copying data is usually the quickest part. The timeline is driven by everything that depends on the data.

What drives the timeline

Driver Why it takes time What to count
Objects in use Every table, view and job needs moving, testing and an owner Tables and views actually queried in the last 90 days
Logic Stored procedures, ETL jobs and scheduled SQL rarely run unchanged on the new platform Procedures, ETL jobs and their complexity
Consumers Every dashboard, extract and application needs new connections and testing Downstream reports, extracts and systems
Access Roles, grants and row-level security have to be recreated Roles and sensitive tables
Proof Each table is reconciled against the old system and signed off by its owner Tables, and how many business owners must sign off
Data volume Bulk copy, mostly parallel Terabytes, but this rarely sets the timeline

The three numbers that explain most timelines are tables in use, procedures and jobs, and downstream consumers. Terabytes rarely do.

The phases of a typical plan

This is how we plan migrations, with typical durations for a mid-sized estate:

Phase Typical duration What happens
Assess 2 to 3 weeks Inventory, usage analysis, wave plan, target architecture and cost model
Pilot wave 4 to 6 weeks One business domain migrated end to end, reconciled and in production
Scale waves 8 to 20 weeks Remaining domains migrated in parallel streams
Decommission 2 to 4 weeks Final consumers switched, legacy system archived and shut down

That adds up to roughly four to eight months. A small estate with little procedural code can be faster. An estate with thousands of procedures, many business owners and strict sign-off takes longer.

Why estimates slip

  1. Unknown consumers. The Excel workbook, the partner extract, the application nobody listed. Every one found late costs days of testing.
  2. Procedural code underestimated. Cursor-heavy procedures and dynamic SQL need rewriting, not translating.
  3. Reconciliation left to the end. Proving the numbers match takes time per table. Done at the end, it all lands at once.
  4. Moving targets. Changes made to the legacy system during the migration have to be made twice.
  5. Big-bang cutover. Switching everything at once concentrates risk and delays. Waves let consumers move one at a time.

How to go faster

  • Retire before you migrate. Usage logs typically show a large share of objects unused. Archive them instead of migrating them.
  • Federate first. Query the old warehouse in place from the new platform, for example with Lakehouse Federation on Databricks, so new work starts on day one while tables move domain by domain.
  • Use tooling for conversion and reconciliation, such as Databricks Labs’ Lakebridge, and test its output closely.
  • Reconcile per wave, not at the end.
  • Freeze the legacy system for changes where the business can accept it.

Estimate your own

Count your tables in use, procedures and jobs, and downstream consumers, then try the migration estimator. It turns those counts into engineer-days, calendar weeks for your team size and a suggested number of waves. For platform-specific plans, see the migration guides, for example Teradata to Databricks or Redshift to Databricks.

The short version

Migrations take months because of logic, consumers, access and proof, not data volume. Count tables in use, procedures and consumers; plan assess, pilot, waves and decommission; retire unused objects first; and reconcile as you go.

Sources

Not sure where your own platform stands? Take the free 3-minute data platform scorecard. Need people to do this alongside your team? See senior data engineers for hire. Unfamiliar terms are explained in the data engineering glossary.

Frequently asked questions

How long does a data warehouse migration take?

For a mid-sized estate, a typical plan is a few weeks of assessment, a pilot wave of four to six weeks, two to five months of further waves, and a few weeks to decommission: roughly four to eight months in total. Large estates with thousands of procedures and many downstream systems take longer. The data volume matters much less than the number of objects, procedures and consumers.

Why does copying the data take so little of the time?

Bulk export and load are mechanical and can run in parallel. The time goes on rewriting logic (stored procedures, ETL jobs, scheduled queries), recreating permissions, repointing every dashboard and downstream system, and proving that the new numbers match the old ones.

How can we make a migration faster?

Retire unused objects first (often a large share), migrate in waves by business domain, use conversion and reconciliation tooling, run old and new in parallel so consumers move one at a time, and freeze changes to the legacy system where you can.

How do I estimate our migration?

Count tables in use, stored procedures and ETL jobs, and downstream consumers (reports, extracts, applications), then apply effort per object type and your team size. The Metric Oak migration estimator does this in the browser.

Have 30 minutes? Get a senior architect’s view of your data platform.

No sales deck. You talk to the engineer who would lead the work.

Book an architecture review
Hemant WalchaleYou talk to Hemant Walchale, founder145 published guidesProfiles: LinkedIn · Clutch · GoodFirms

Free, takes 20 seconds

Get your starter data platform roadmap

Pick your platform and your main goal. You get a 90-day plan written by senior data engineers: what to do first, next and after.

You’ll also get our monthly email (confirm it in your inbox). Unsubscribe any time. Privacy policy.