We are seeking a highly skilled Senior Kafka Engineer to design, build, optimize, and support large-scale event-driven platforms leveraging Apache Kafka. This role will be responsible for architecting high-throughput, low-latency streaming solutions, ensuring reliable message delivery, optimizing platform performance, and enabling real-time data integration across enterprise systems. The ideal candidate has deep expertise in Kafka architecture, Java development, event streaming, and distributed systems.
Key Responsibilities:
- Design and implement scalable, fault-tolerant event-driven architectures using Apache Kafka and Confluent Platform
- Lead Kafka topic and partition strategy design, ensuring optimal performance, scalability, and message ordering requirements
- Develop and maintain producer and consumer applications using Java
- Configure and manage consumer groups, offset management, partition assignments, and rebalancing strategies
- Design and implement robust retry, dead-letter queue (DLQ), replay, and error-handling frameworks
- Establish and enforce schema governance using Schema Registry, including Avro and Protobuf serialization standards
- Build and support enterprise event-routing and streaming data-plane solutions that facilitate reliable real-time data movement across systems
- Administer and secure Kafka environments through ACL management, authentication, and authorization controls
- Monitor, troubleshoot, and optimize Kafka clusters, including performance tuning, throughput optimization, and consumer lag reduction
- Collaborate with application, infrastructure, and data engineering teams to define enterprise event streaming standards and best practices
- Support production environments, investigate incidents, and implement proactive monitoring and observability solutions
Requirements
- 7+ years of software engineering experience with at least 4+ years focused on Apache Kafka
- Expert-level knowledge of Kafka architecture, including brokers, topics, partitions, replication, consumer groups, and message delivery semantics
- Hands-on experience with Confluent Platform (preferred)
- Strong experience designing and implementing event-driven and streaming architectures
- Advanced proficiency in Java development
- Experience with Schema Registry, Avro, and Protobuf
- Deep understanding of message ordering, partitioning strategies, replay mechanisms, and DLQ patterns
- Experience implementing and managing Kafka security, including ACLs, authentication, and authorization
- Strong troubleshooting and performance tuning skills, including consumer lag analysis and optimization
- Experience working with distributed systems in cloud or hybrid enterprise environments
- Strong understanding of high-availability and disaster recovery patterns for streaming platforms
Preferred Qualifications:
- Confluent Certified Developer or Administrator certification
- Experience with Kubernetes, Docker, and cloud platforms (AWS, Azure, or GCP)
- Familiarity with Kafka Connect, ksqlDB, MirrorMaker, and stream processing frameworks
- Experience with observability tools such as Prometheus, Grafana, Splunk, or Datadog
- Knowledge of CI/CD pipelines and Infrastructure as Code (Terraform, Ansible, etc.)
Technical Skills:
- Apache Kafka
- Confluent Platform
- Java
- Schema Registry
- Avro/Protobuf
- Kafka ACLs
- Consumer Groups
- Topic & Partition Design
- DLQ/Retry/Replay Patterns
- Event-Driven Architecture
- Streaming Data Platforms
- Performance Tuning & Consumer Lag Optimization
- Distributed Systems
- Kubernetes/Cloud Platforms (preferred)
-
Are you interested in this position?
Apply by clicking on the “Apply Now” button below!
#AlbionarcJobs#FintechJobs
#AsiaJobs#MiddleEastCareers
#TechTalent#FintechRecruitment
#FinanceOpportunities#
