As a Data Engineer, you will build and optimize the data infrastructure that powers the platform’s analytics capabilities. You’ll design scalable data pipelines, develop distributed processing jobs, and ensure high-quality data is available to support AI-driven anomaly detection and advanced analytics. This role is ideal for engineers who enjoy working with large datasets, cloud technologies, and modern data engineering frameworks.
Responsibilities:
- Design, build, and maintain scalable data pipelines that ingest telemetry, log, and configuration data
- Develop distributed data processing applications using Apache Spark
- Build and optimize ETL/ELT workflows within a cloud-based data lake
- Collaborate with Data Scientists and AI Engineers to prepare and deliver data for analytics and machine learning workloads
- Optimize pipeline performance, scalability, and reliability
- Ensure data quality through monitoring, validation, and troubleshooting
- Partner with Platform Engineering to deploy and support data processing applications in AWS
- Contribute to data architecture decisions and engineering best practices
Requirements
- 5+ years of experience in Data Engineering or a related field
- Strong experience developing distributed processing applications with Apache Spark
- Experience with a second programming language such as Scala (preferred), Java, or another compiled language
- Experience designing and building scalable ETL/ELT pipelines
- Hands-on experience working with AWS cloud services
- Experience with cloud-based data lakes and large-scale datasets
- Strong understanding of distributed systems and data engineering best practices
- Proficiency in Python
Preferred Qualifications:
- Experience working with telemetry, log, or operational data
- Experience supporting AI, machine learning, or anomaly detection initiatives
- Experience working in Agile development environments
- Familiarity with streaming data technologies
- Knowledge of graph-based data models or network-related data
-
Are you interested in this position?
Apply by clicking on the “Apply Now” button below!
#AlbionarcJobs#FintechJobs
#AsiaJobs#MiddleEastCareers
#TechTalent#FintechRecruitment
#FinanceOpportunities#
