Migration
Amazon Redshift to Databricks migration: a practical guide
How to move Amazon Redshift to Databricks on AWS: unloading to S3 as Delta, Spectrum tables into Unity Catalog, converting SQL and procedures, Lakehouse Federation for a gradual cutover, and replacing WLM with SQL warehouses.
Hemant Walchale
· 2 min read
Redshift customers move to Databricks to bring analytics, data engineering and machine learning onto one platform, to stop managing cluster sizes and WLM queues, and to keep data in open formats on S3. Both run on AWS, so data movement is simple. What makes a Redshift migration smooth is moving gradually rather than all at once.
This is the approach we recommend.
1. Inventory
List schemas, tables, views, materialized views, stored procedures, UDFs, scheduled queries, Spectrum external tables, load jobs and BI consumers. Redshift system tables show which objects are actually queried. Archive the rest.
2. Federate first, then migrate
Connect Redshift to Databricks with Lakehouse Federation. New work can start on Databricks immediately, reading Redshift tables in place, while you migrate tables domain by domain. Each migrated table replaces its federated version in the same catalog, so consumers don’t need to know which is which.
3. Map Redshift to Databricks
| Redshift | On Databricks |
|---|---|
| Tables | Delta tables in Unity Catalog (unload to S3 as Parquet, then load or convert) |
| Spectrum external tables | Unity Catalog external tables, or CONVERT TO DELTA |
DISTKEY, DISTSTYLE |
Not needed |
SORTKEY |
Liquid clustering |
VACUUM, ANALYZE |
Predictive optimization or scheduled OPTIMIZE |
SUPER and PartiQL |
VARIANT type or structs and arrays, with Spark SQL path syntax |
| PL/pgSQL stored procedures | Set-based SQL, or Python tasks |
| WLM queues, concurrency scaling | Separate (often serverless) SQL warehouses per workload |
| Scheduled queries | Lakeflow Jobs |
| Materialized views | Materialized views in Databricks SQL, or Lakeflow Declarative Pipelines |
| Grants and row-level security | Unity Catalog grants, row filters and column masks |
4. Convert SQL and reconcile
Most Redshift SQL runs on Databricks SQL with small changes; test date functions, string functions, implicit casts and VARCHAR semantics (Databricks STRING has no length limit). Compare every migrated table against Redshift by domain before switching consumers over.
5. Shrink and retire clusters
Resize Redshift down as domains move, then pause and delete clusters, and let reserved instances expire rather than renewing them.
The short version
Federate Redshift into Unity Catalog so new work can start at once, move tables to Delta through S3 domain by domain, replace sort keys with liquid clustering and WLM with separate SQL warehouses, rewrite procedures, reconcile and shut clusters down as you go.
Prefer Snowflake? See Redshift to Snowflake. Estimate costs with the cost calculator or browse all migration guides.
Not sure where your own platform stands? Take the free 3-minute data platform scorecard, or see how we work on Databricks. Unfamiliar terms are explained in the data engineering glossary.