Our Client Currently looking for Data Engineer (Snowflake to BigQuery Migration)

What you will do

Transformation migration — port the Snowflake transformation layer (Streams + Tasks CDC, stored procedures, dynamic tables) to BigQuery — primarily incremental models. SQL translation — translate Snowflake SQL and script-based jobs into dbt models and macros, using the BigQuery Migration Service for the bulk translation plus manual fixes for what does not auto-translate.

Re-architecture (as required) — re-architect constructs with no direct BigQuery equivalent (Streams, dynamic tables, zero-copy clone, JavaScript stored procedures, usage-metadata jobs) into native BigQuery patterns or Cloud Run jobs.

Ingestion rebuild — move Kafka ingestion to the BigQuery Sink connector on the existing Kubernetes footprint; replace the current managed ELT tool with open-source (Airbyte OSS etc) / managed CDC; switch BI and event sources to native BigQuery destinations.

BigQuery foundation & security — design datasets, regions (including data-residency boundaries), partitioning and clustering; implement IAM, row-level access policies, and column-level controls / authorized views.

Orchestration (TBD) — build Cloud Composer (Airflow) DAGs — or Cloud Scheduler + Workflows — to replace legacy cron-based scheduling, with dependencies, retries, backfills, and alerting.

Historical data migration — run the one-time historical backfill using unload-to-GCS loads and the BigQuery Data Transfer Service, applying the right partition/cluster design as data lands.

Validation & cutover — run BigQuery and Snowflake in parallel, reconcile results, repoint downstream consumers, and execute the freeze / final-delta / cutover.

Required skills

These capabilities most directly determine whether the project succeeds. A strong candidate is genuinely hands-on across both the transformation and platform sides.

BigQuery (expert)
Deep, hands-on BigQuery: Standard SQL, partitioning & clustering design, IAM, row-level access policies, policy tags / column masking, authorized views, on-demand vs. slot reservations (Editions / autoscaling), Storage Write API, load jobs, and the BigQuery Migration Service + Data Transfer Service.

Snowflake (Intermediate)
Practical experience with the Snowflake internals being migrated away from: Streams, Tasks, Snowpipe, stored procedures, dynamic tables, zero-copy clone, RBAC, and row-access policies.

SQL dialect translation
Fluent translation between Snowflake and BigQuery SQL, including semi-structured / VARIANT ↔ JSON/STRUCT handling, and the judgment to know what will not auto-translate.

Data ingestion / CDC
Kafka Connect (BigQuery Sink / Storage Write API), plus at least one of Airbyte, Datastream, or comparable open-source / managed CDC.

Orchestration (TBD)
Cloud Composer / Apache Airflow (DAG design, retries, backfills) — or Cloud Scheduler + Workflows — replacing legacy cron-based scheduling.

Python
Solid Python for data engineering: porting connector-based batch jobs and building Cloud Run jobs for non-SQL logic.

GCP fundamentals
Service accounts / Workload Identity, GCS, billing & cost modeling, and reasoning about scan-based vs. reservation pricing to actually realize the cost savings.

Migration & reconciliation
Track record on large-table backfills and data validation: row-count / aggregate reconciliation, delta sync, and freeze/cutover execution with minimal downtime.

Preferred (nice to have)

• Kubernetes tooling: GKE, ArgoCD, Helm, and Terraform — the ingestion connectors deploy this way.
• BI tooling migration (e.g., Looker / LookML) — repointing connections and PDT strategy.
• Data governance and PII handling: policy tags, column masking, and preserving anonymization logic.
• Event-streaming and reverse-ETL platforms with native BigQuery support.
• Experience decommissioning a managed ELT tool.
• Prior end-to-end warehouse migration (Snowflake, Redshift, or Teradata → BigQuery) delivered through cutover.

• Are you interested in this position?

   

  Apply by clicking on the “Apply Now” button below!

   

  #AlbionarcJobs#FintechJobs

  #AsiaJobs#MiddleEastCareers

  #TechTalent#FintechRecruitment

  #FinanceOpportunities#