Migration
Google BigQuery to Databricks migration: a practical guide
How to migrate BigQuery to Databricks: exporting tables to Delta, converting GoogleSQL to Spark SQL (UNNEST, STRUCT, SAFE functions), partitioned and clustered tables, scheduled queries, BigQuery ML and Lakehouse Federation.
Hemant Walchale
· 2 min read
Teams move from BigQuery to Databricks to run data engineering, machine learning and analytics on one platform, to keep data in open Delta format, and to get predictable control over compute costs. Databricks runs on Google Cloud, so the move can stay in the same region. Nested and repeated data is the area that needs the most care.
This is the approach we recommend.
1. Inventory
List datasets, tables (partitioned and clustered), views, authorized views, routines, scheduled queries, Dataform or dbt projects, BigQuery ML models and the dashboards that read them. INFORMATION_SCHEMA.JOBS shows what is queried and by whom; retire the rest.
2. Federate first
Connect BigQuery to Unity Catalog with Lakehouse Federation, so new work on Databricks can read BigQuery tables immediately while you migrate them domain by domain.
3. Map BigQuery to Databricks
| BigQuery | On Databricks |
|---|---|
| Datasets and tables | Schemas and Delta tables in Unity Catalog |
| Partitioned and clustered tables | Liquid clustering (partition only very large tables by date if needed) |
STRUCT, ARRAY, nested and repeated fields |
STRUCT, ARRAY; UNNEST becomes explode or LATERAL VIEW |
SAFE_CAST, SAFE_DIVIDE and other SAFE. functions |
try_cast, try_divide and other try_ functions |
QUALIFY |
Supported |
| Scheduled queries | Lakeflow Jobs |
| Dataform or dbt | dbt on Databricks, or Lakeflow Declarative Pipelines |
| Authorized views, row-level security | Unity Catalog views, row filters and column masks |
| BigQuery ML | Python with MLflow, model serving |
| Slots and reservations | SQL warehouses and job compute, sized per workload |
4. Export data and reconcile
Export tables to Google Cloud Storage as Parquet (with EXPORT DATA), load or convert them to Delta, and keep them in sync for the parallel run. Compare every table by domain, paying attention to nested fields, TIMESTAMP versus DATETIME semantics and numeric types (NUMERIC, BIGNUMERIC).
5. Cut over and reduce BigQuery spend
Repoint dashboards domain by domain, then reduce reservations or on-demand usage as the workload moves.
The short version
Federate BigQuery into Unity Catalog, export tables to Delta on Cloud Storage, convert nested-data SQL and SAFE functions, replace scheduled queries and BigQuery ML with Lakeflow Jobs and MLflow, reconcile carefully on nested fields and reduce BigQuery spend domain by domain.
Browse all migration guides, compare platforms with the platform chooser, or estimate costs with the cost calculator.
Not sure where your own platform stands? Take the free 3-minute data platform scorecard, or see how we work on Databricks. Unfamiliar terms are explained in the data engineering glossary.