Our Client is looking for a Lead Data Software Engineer with expertise in data platform, data lakes, Lakehouse architectures, data warehouses, and delta lakes to drive our modern data infrastructure. This role will focus on enabling data science teams, integrating ML and LLM solutions, and building scalable, event-driven data platforms using cutting-edge AWS services.
Essential Functions :
Data Engineering & Data Architectures :
– Design and implement data lake, Lakehouse, and data warehouse architectures leveraging AWS Data Lake Formation, Redshift, and Delta Lakes
– Build and maintain scalable ETL pipelines using AWS Glue, Apache Spark, Databricks, and EMR
– Develop data ingestion, transformation, and enrichment workflows using Python, Spark, and SQL
– Optimize data storage and partitioning strategies (Parquet, Delta, Iceberg) for performance and cost efficiency
– Implement real-time and batch data processing frameworks to support analytics and AI-driven use cases
– Leverage serverless computing (AWS Lambda, Fargate) and containerized compute (ECS, EKS, Kubernetes) to scale data workloads
ML & LLM Integration :
– Integrate machine learning (ML) and large language model (LLM) solutions into production data pipelines
– Utilize AWS SageMaker, Databricks ML, Bedrock, and Redshift ML to support AI/ML workloads
– Apply MLOps frameworks to manage model deployment, monitoring, and retraining at scale
AI-Assisted Development :
– Incorporate AI coding tools (such as Claude Code, GitHub Copilot, Cursor, or equivalent) into daily development workflows to accelerate delivery
– Effectively prompt, review, and validate AI-generated code across Python, SQL, and Spark workloads
– Integrate AI tools into CI/CD pipelines to improve code quality, reduce cycle time, and increase sprint velocity
– Track and report on AI tool ROI using metrics such as story point reduction, PR throughput, and cycle time improvement
– Model responsible AI-assisted development practices, including thorough code review, testing, and validation of AI-generated outputs
DevOps, CI/CD & Automation :
– Uphold and advance DevOps best practices across application development and deployment workflows
– Containerize and orchestrate data workloads using Docker, Kubernetes, AWS ECS, and EKS
– Drive automated testing integration including unit, integration, performance, and security testing into DevOps pipelines
– Monitor system health and data platform observability using AWS CloudWatch, Datadog, and OpenTelemetry
Leadership & Collaboration :
– Mentor and lead data engineering teams in building and optimizing modern data platforms
– Partner with data science, AI/ML, and business analytics teams to drive data-driven innovation across the organization
– Align technical strategies with business goals, ensuring solutions meet scalability, governance, and compliance requirements
– Communicate technical concepts clearly to engineering peers, data science stakeholders, and executive leadership
– Champion AI-assisted development best practices across the team and coach engineers on effective AI tool usage
Education Requirement : Bachelors degree in a related field or equivalent education and work experience.
Required Experience, Knowledge and Skills :
– 8+ years of experience in data engineering, cloud architectures, and ML/AI integrations
– Hands-on experience with Databricks, Delta Lake, AWS Redshift, and modern data Lakehouse solutions
– Demonstrated use of AI development tools to improve personal and team productivity
– AWS certifications (Solutions Architect, or equivalent)
Are you interested in this position?
Apply by clicking on the “Apply Now” button below!
#AlbionarcJobs#FintechJobs
#AsiaJobs#MiddleEastCareers
#TechTalent#FintechRecruitment
#FinanceOpportunities#
