We are looking for a skilled Module Lead – Big Data with strong experience in Big Data and Data Engineering. The candidate should have hands-on expertise in building scalable data pipelines, ETL/ELT processes, cloud-based data platforms, and modern data engineering technologies.

Key Responsibilities :

– Use the latest technologies to build data pipelines and integrate machine learning models.

– Build and expand the enterprise data platform.

– Develop applications that run on Google Cloud-based infrastructure.

– Design and develop data pipelines and ETL/ELT processes.

– Work with advanced SQL and data warehousing concepts including BigQuery.

– Develop solutions using :

i. Python

ii. PySpark

iii. SQL

– Work with relational and NoSQL databases including :

i. PostgreSQL

ii. MongoDB

– Use workflow management tools such as :

i. Airflow

ii. AWS Data Pipeline

iii. Google Cloud Composer

– Build Docker images and deploy them to production environments.

– Integrate Docker container orchestration frameworks using Kubernetes.

– Create :

i. Pods

ii. ConfigMaps

iii. Deployments

using Terraform.

– Work with Azure DevOps or similar CI/CD tools and Git.

Required Skills & Experience :

– 6+ years of experience in :

i. Big Data

ii. Data Engineering

– Strong knowledge of :

i. Advanced SQL

ii. Data Warehousing concepts

iii. BigQuery

– Strong programming skills in :

i. SQL

ii. Python

iii. PySpark

– Experience with :

i. Data pipeline development

ii. ETL/ELT processes

– Experience working with one of the public cloud providers :

i. GCP

ii. Azure

iii. AWS

– Experience with relational SQL and NoSQL databases.

– Experience with workflow management and orchestration tools.

– Hands-on experience with :

i. Docker

ii. Kubernetes

iii. Terraform

– Familiarity with :

i. Azure DevOps

ii. CI/CD tools

iii. Git

– Strong problem-solving and communication skills.

Nice to Have :

– Experience with :

i. C#

ii. .NET Core

iii. Microservices

Education :

– Bachelor’s degree or advanced degree in Computer Science or related engineering discipline.

Key Skills :

– Python

– PySpark

– Data Engineering

– ETL

– SQL

– BigQuery

– Kubernetes

– Docker

– Terraform

– GCP

– Airflow

– MongoDB

Are you interested in this position?

Apply by clicking on the “Apply Now” button below!

#AlbionarcJobs#FintechJobs

#AsiaJobs#MiddleEastCareers

#TechTalent#FintechRecruitment

#FinanceOpportunities#

L