Migration
Greenplum to Databricks migration: a practical guide
How to migrate a Greenplum MPP warehouse to Databricks: converting PostgreSQL-based SQL and PL/pgSQL, replacing DISTRIBUTED BY and append-optimized tables, gpfdist loads, MADlib analytics, and reconciliation.
Hemant Walchale
· 2 min read
Greenplum is a massively parallel warehouse built on PostgreSQL, common in telecoms, finance and the public sector. Changes to its licensing and open-source status in recent years have pushed many Greenplum users to plan a move. Databricks is a common destination, especially where analytics written with MADlib or Python need a modern home.
This is the approach we recommend.
1. Inventory
List databases, schemas, tables (heap, append-optimized, column-oriented), views, PL/pgSQL functions, external tables, gpload and gpfdist jobs, MADlib models, schedules and consumers. Use pg_stat views and query logs to find what is used; archive the rest.
2. Map Greenplum to Databricks
| Greenplum | On Databricks |
|---|---|
| Tables (heap, AO, AOCO) | Delta tables in Unity Catalog |
DISTRIBUTED BY, DISTRIBUTED RANDOMLY |
Not needed |
| Partitioned tables | Liquid clustering; partition only very large tables by date if needed |
VACUUM, ANALYZE |
Predictive optimization or scheduled OPTIMIZE |
| PostgreSQL-dialect SQL | Databricks SQL; test functions, casts and operators |
| PL/pgSQL functions | Set-based SQL or Python |
| gpfdist, gpload, external tables | Auto Loader, COPY INTO, external tables |
| MADlib | Spark MLlib, scikit-learn and MLflow |
| Roles and grants | Unity Catalog groups, grants, row filters and column masks |
| cron or other schedulers | Lakeflow Jobs |
3. Convert SQL carefully
Most queries move with small changes. Watch for PostgreSQL casts (::type), array and JSON functions, regular expression operators, DISTINCT ON, sequences and serial columns (use identity columns), and implicit type coercion. Test numeric precision and timestamp handling.
4. Move data and reconcile
Export tables in parallel to cloud storage (for example with writable external tables or unload jobs), load them as Delta, and keep them in sync during the parallel run. Compare every table by domain and cut over when owners sign off.
5. Retire the cluster
Stop each domain’s Greenplum jobs on cutover day and decommission segment hosts as the cluster empties.
The short version
Archive what’s unused, load tables as Delta without distribution keys, convert PostgreSQL-dialect SQL and PL/pgSQL, move MADlib models to Python with MLflow, replace gpfdist with Auto Loader, reconcile and retire the cluster by domain.
Browse all migration guides, or estimate effort with the migration estimator.
Not sure where your own platform stands? Take the free 3-minute data platform scorecard, or see how we work on Databricks. Unfamiliar terms are explained in the data engineering glossary.