Our Client Currently looking for Lead Data Software Engineer
Essential Functions :
Data Engineering & Data Architectures :
– Design and implement data lake, Lakehouse, and data warehouse architectures leveraging AWS Data Lake Formation, Redshift, and Delta Lakes
– Build and maintain scalable ETL pipelines using AWS Glue, Apache Spark, Databricks, and EMR
– Develop data ingestion, transformation, and enrichment workflows using Python, Spark, and SQL
– Optimize data storage and partitioning strategies (Parquet, Delta, Iceberg) for performance and cost efficiency
– Implement real-time and batch data processing frameworks to support analytics and AI-driven use cases
– Leverage serverless computing (AWS Lambda, Fargate) and containerized compute (ECS, EKS, Kubernetes) to scale data workloads
ML & LLM Integration :
– Integrate machine learning (ML) and large language model (LLM) solutions into production data pipelines
– Utilize AWS SageMaker, Databricks ML, Bedrock, and Redshift ML to support AI/ML workloads
– Apply MLOps frameworks to manage model deployment, monitoring, and retraining at scale
AI-Assisted Development :
– Incorporate AI coding tools (such as Claude Code, GitHub Copilot, Cursor, or equivalent) into daily development workflows to accelerate delivery
– Effectively prompt, review, and validate AI-generated code across Python, SQL, and Spark workloads
– Integrate AI tools into CI/CD pipelines to improve code quality, reduce cycle time, and increase sprint velocity
– Track and report on AI tool ROI using metrics such as story point reduction, PR throughput, and cycle time improvement
– Model responsible AI-assisted development practices, including thorough code review, testing, and validation of AI-generated outputs
DevOps, CI/CD & Automation :
– Uphold and advance DevOps best practices across application development and deployment workflows
– Containerize and orchestrate data workloads using Docker, Kubernetes, AWS ECS, and EKS
– Drive automated testing integration including unit, integration, performance, and security testing into DevOps pipelines
– Monitor system health and data platform observability using AWS CloudWatch, Datadog, and OpenTelemetry
Leadership & Collaboration :
– Mentor and lead data engineering teams in building and optimizing modern data platforms
– Partner with data science, AI/ML, and business analytics teams to drive data-driven innovation across the organization
– Align technical strategies with business goals, ensuring solutions meet scalability, governance, and compliance requirements
– Communicate technical concepts clearly to engineering peers, data science stakeholders, and executive leadership
– Champion AI-assisted development best practices across the team and coach engineers on effective AI tool usage
Education Requirement : Bachelors degree in a related field or equivalent education and work experience.
Required Experience, Knowledge and Skills :
– 8+ years of experience in data engineering, cloud architectures, and ML/AI integrations
– Hands-on experience with Databricks, Delta Lake, AWS Redshift, and modern data Lakehouse solutions
– Demonstrated use of AI development tools to improve personal and team productivity
– AWS certifications (Solutions Architect, or equivalent)
Are you interested in this position?
Apply by clicking on the “Apply Now” button below!
#AlbionarcJobs#FintechJobs
#AsiaJobs#MiddleEastCareers
#TechTalent#FintechRecruitment
#FinanceOpportunities#
