Migration
Talend to Databricks migration: converting Talend jobs to Spark SQL and Lakeflow
How to migrate Talend Studio jobs, including Talend Big Data Spark jobs, to Databricks: inventory components, map tMap and other components to Spark SQL, rewrite tJava code, and replace contexts and scheduling with Lakeflow Jobs.
Hemant Walchale
· 2 min read
Talend jobs are a common sight in front of a Databricks lakehouse: extracting from source systems, transforming in Java, and writing to Delta. Moving that logic into Databricks removes a separate ETL engine and licence, and puts transformation, quality checks and governance in one place.
This is the approach we recommend.
1. Inventory
Talend jobs are stored as XML .item files, one node per component. Count the components across the estate:
grep -rhoE 'componentName="[^"]+"' --include=*.item . | sort | uniq -c | sort -rn
Separate standard (Java) jobs from Big Data Spark jobs, and collect context groups, joblets, routines, child jobs, schedules and run history. Retire jobs that no longer run.
2. Map components to Databricks
| Talend component | On Databricks |
|---|---|
tDBInput, tFileInput*, SaaS connectors |
Lakeflow Connect, Auto Loader or JDBC reads into bronze Delta tables |
tMap |
Joins, SELECT expressions, WHERE, and one write per output |
tFilterRow, tJoin, tAggregateRow |
WHERE, JOIN, GROUP BY |
tSortRow, tUniqRow |
Window functions such as ROW_NUMBER() |
tNormalize, tDenormalize |
explode, collect_list |
tDBOutput with upsert |
MERGE INTO, or AUTO CDC in Lakeflow Declarative Pipelines |
| SCD components | AUTO CDC with SCD type 2 |
tJava, tJavaRow, routines |
SQL functions or Python; rewrite and test |
tRunJob, joblets |
Task dependencies in Lakeflow Jobs, shared SQL or Python modules |
| Context variables | Job parameters and secret scopes |
| TAC or Talend Cloud schedules | Lakeflow Jobs schedules and file-arrival triggers |
tLogCatcher, tStatCatcher |
Job run history, system tables and expectations |
3. Rebuild as layered pipelines
Organize the logic as bronze, silver and gold layers in Lakeflow Declarative Pipelines rather than one-for-one copies of each job. Expectations on each table replace the hand-built reject flows common in Talend jobs.
4. Reconcile and cut over
Run old and new side by side on the same sources, compare every target table, and watch Java-versus-SQL differences in nulls, string handling, rounding and date parsing. Cut over by domain and disable the matching Talend schedules the same day.
The short version
Count components from the job XML, start with Spark jobs, translate tMap and friends into Spark SQL, rewrite Java code deliberately, replace contexts with job parameters and secrets, rebuild as layered pipelines with expectations, reconcile and switch off Talend by domain.
Moving to Snowflake instead? See Talend to Snowflake. Browse all migration guides or estimate effort with the migration estimator.
Not sure where your own platform stands? Take the free 3-minute data platform scorecard, or see how we work on Databricks. Unfamiliar terms are explained in the data engineering glossary.