33+ Apache Kafka Jobs in Delhi, NCR and Gurgaon | Apache Kafka Job openings in Delhi, NCR and Gurgaon

Egen is a data engineering and cloud modernization firm helping industry-leading companies achieve digital breakthroughs and deliver for the future, today. We are catalysts for change who create digital breakthroughs at warp speed. Our team of cloud and data engineering experts are trusted by top clients in pursuit of the extraordinary. An Inc. 5000 Fastest Growing Company 7 times, and recently recognized on the Crain’s Chicago Business Fast 50 list, Egen has also been recognized as a great place to work 3 times.

You will join a team of insatiably curious data engineers, software architects, and product experts who never settle for "good enough". Our Java Platform team's tech stack is based on Java8 (Spring Boot) and RESTful web services. We typically build and deploy applications as cloud-native Kubernetes microservices and integrate with scalable technologies such as Kafka in Docker container environments. Our developers work in an agile process to efficiently deliver high value data driven applications and product packages.

Required Experience:

• Minimum of Bachelor’s Degree or its equivalent in Computer Science, Computer Information Systems, Information Technology and Management, Electrical Engineering or a related field.
• Have experience working and strong understanding of object-oriented programing and cloud technologies
• End to end experience delivering production ready code with Java8, Spring Boot, Spring Data, and API libraries
• Strong experience with unit and integration testing of the Spring Boot APIs.
• Strong understanding and production experience of RESTful API's and microservice architecture.
• Strong understanding of SQL databases and NoSQL databases and experience with writing abstraction layers to communicate with the databases.

Nice to have's (but not required):

• Exposure to Kotlin or other JVM programming languages
• Strong understanding and production experience working with Docker container environments
• Strong understanding and production experience working with Kafka
• Cloud Environments: AWS, GCP or Azure

Key Responsibilities:

● Set up a production-grade Apache Kafka in Kubernetes with Jenkins integration.

● Design a Jenkins-based CI/CD pipeline for multiple components, including Kafka Connect.

● Implement Horizontal Pod Autoscaler (HPA) for our 30-40 micro components.

● Configure comprehensive logging and monitoring, setting up Grafana dashboards with

Prometheus integration.

Key Requirements:

Must-Have Skills:

● Deep expertise in Git, Jenkins, Kubernetes, Helm, Docker, Kafka, Kafka Connect, Maven, Java,

Prometheus, Grafana, EKS/AKS, Load Balancer, and Ingress.

● Experience working with AWS and Azure cloud environments.

● 5+ years of relevant work experience.

Nice to Have:

● Familiarity with managing On-prem Kubernetes is a plus.

● Familiarity with Secret Manager, NodeJs, Python, Confluent Kafka, MongoDB, ElasticSearch,

FluentBit, FluentD, Aerospike, HBase, and MySQL.

● If you’re passionate about orchestrating large-scale system migrations and love to stay on the

bleeding edge of technology, then you’re just the professional we’re looking for.

Role : Senior Engineer Infrastructure

Key Responsibilities:

● Infrastructure Development and Management: Design, implement, and manage robust and scalable infrastructure solutions, ensuring optimal performance,security, and availability. Lead transition and migration projects, moving legacy systemsto cloud-based solutions.

● Develop and maintain applications and services using Golang.

● Automation and Optimization: Implement automation tools and frameworksto optimize operational processes. Monitorsystem performance, optimizing and modifying systems as necessary.

● Security and Compliance: Ensure infrastructure security by implementing industry best practices and compliance requirements. Respond to and mitigate security incidents and vulnerabilities.

Qualifications:

● Bachelor's degree in Computer Science, Engineering, or a related field (or equivalent practical experience).

● Good understanding of prominent backend languageslike Golang, Python, Node.js, or others.

● In-depth knowledge of network architecture,system security, infrastructure scalability.

● Proficiency with development tools,server management, and database systems.

● Strong experience with cloud services(AWS.), deployment,scaling, and management.

● Knowledge of Azure is a plus

● Familiarity with containers and orchestration services,such as Docker, Kubernetes, etc.

● Strong problem-solving skills and analytical thinking.

● Excellent verbal and written communication skills.

● Ability to thrive in a collaborative team environment.

● Genuine passion for backend development and keen interest in scalable systems. 

Required Skills:

• 3+ year of experience in Development in JAVA technology.
• Strong Java Basics
• SpringBoot or Spring MVC
• Hands on experience on Relationl Databases (SQL query or Hibernate) + Mongo (JSON parsing)
• Proficient in REST API development
• Messaging Queue (RabitMQ or Kafka)
• Microservices
• Any Caching Mechanism
• Good at problem solving

Good to Have Skills:

• 4+ years of experience in using Java/J2EE tech stacks
• Good understanding of data structures and algorithms.
• Excellent analytical and problem solving skills.
• Ability to work in a fast paced internet start-up environment.
• Experience in technical mentorship/coaching is highly desirable.
• Understanding of AI/ML algorithms is a plus.

Job Title: Data Engineer

Job Summary: As a Data Engineer, you will be responsible for designing, building, and maintaining the infrastructure and tools necessary for data collection, storage, processing, and analysis. You will work closely with data scientists and analysts to ensure that data is available, accessible, and in a format that can be easily consumed for business insights.

Responsibilities:

• Design, build, and maintain data pipelines to collect, store, and process data from various sources.
• Create and manage data warehousing and data lake solutions.
• Develop and maintain data processing and data integration tools.
• Collaborate with data scientists and analysts to design and implement data models and algorithms for data analysis.
• Optimize and scale existing data infrastructure to ensure it meets the needs of the business.
• Ensure data quality and integrity across all data sources.
• Develop and implement best practices for data governance, security, and privacy.
• Monitor data pipeline performance / Errors and troubleshoot issues as needed.
• Stay up-to-date with emerging data technologies and best practices.

Requirements:

Bachelor's degree in Computer Science, Information Systems, or a related field.

Experience with ETL tools like Matillion,SSIS,Informatica

Experience with SQL and relational databases such as SQL server, MySQL, PostgreSQL, or Oracle.

Experience in writing complex SQL queries

Strong programming skills in languages such as Python, Java, or Scala.

Experience with data modeling, data warehousing, and data integration.

Strong problem-solving skills and ability to work independently.

Excellent communication and collaboration skills.

Familiarity with big data technologies such as Hadoop, Spark, or Kafka.

Familiarity with data warehouse/Data lake technologies like Snowflake or Databricks

Familiarity with cloud computing platforms such as AWS, Azure, or GCP.

Familiarity with Reporting tools

Teamwork/ growth contribution

• Helping the team in taking the Interviews and identifying right candidates
• Adhering to timelines
• Intime status communication and upfront communication of any risks
• Tech, train, share knowledge with peers.
• Good Communication skills
• Proven abilities to take initiative and be innovative
• Analytical mind with a problem-solving aptitude

Good to have :

Master's degree in Computer Science, Information Systems, or a related field.

Experience with NoSQL databases such as MongoDB or Cassandra.

Familiarity with data visualization and business intelligence tools such as Tableau or Power BI.

Knowledge of machine learning and statistical modeling techniques.

If you are passionate about data and want to work with a dynamic team of data scientists and analysts, we encourage you to apply for this position.

• 3+ year of experience in Development in JAVA technology.
• Strong Java Basics
• SpringBoot or Spring MVC
• Hands on experience on Relationl Databases (SQL query or Hibernate) + Mongo (JSON parsing)
• Proficient in REST API development
• Messaging Queue (RabitMQ or Kafka)
• Microservices
• Any Caching Mechanism
• Good at problem solving

Good to Have Skills:

• 4+ years of experience in using Java/J2EE tech stacks
• Good understanding of data structures and algorithms.
• Excellent analytical and problem solving skills.
• Ability to work in a fast paced internet start-up environment.
• Experience in technical mentorship/coaching is highly desirable.
• Understanding of AI/ML algorithms is a plus.

Required Education:

B.Tech./ BE - Computer, IT, Electronics only

Required Skills:

• 2+ year of experience in Development in JAVA technology.
• Strong Java Basics
• SpringBoot or Spring MVC
• Hands on experience on Relationl Databases (SQL query or Hibernate) + Mongo (JSON parsing)
• Proficient in REST API development
• Messaging Queue (RabitMQ or Kafka)
• Microservices
• Any Caching Mechanism
• Good at problem solving

Good to Have Skills:

• 4+ years of experience in using Java/J2EE tech stacks
• Good understanding of data structures and algorithms.
• Excellent analytical and problem solving skills.
• Ability to work in a fast paced internet start-up environment.
• Experience in technical mentorship/coaching is highly desirable.
• Understanding of AI/ML algorithms is a plus

Requirements

• 2+ years of experience in the Development of JAVA technology.
• Strong Java Basics
• Linux
• SpringBoot or Spring MVC
• Hands-on experience in Relational Databases (SQL query or Hibernate) + Mongo (JSON parsing)
• Proficient in REST API development
• Messaging Queue (RabitMQ or Kafka)
• Microservices
• Java 8
• Any Caching Mechanism
• Good at problem-solving

Good to Have Skills:

• 2+ years of experience in using Java/J2EE tech stacks
• Good understanding of data structures and algorithms.
• Excellent analytical and problem-solving skills.
• Ability to work in a fast-paced internet start-up environment.
• Experience in technical mentorship/coaching is highly desirable.
• Understanding AI/ML algorithms is a plus.

Requirements

• 3+ years of experience in the Development of JAVA technology.
• Strong Java Basics
• Linux
• SpringBoot or Spring MVC
• Hands-on experience in Relational Databases (SQL query or Hibernate) + Mongo (JSON parsing)
• Proficient in REST API development
• Messaging Queue (RabitMQ or Kafka)
• Microservices
• Java 8
• Any Caching Mechanism
• Good at problem-solving

Good to Have Skills:

• 3 years of experience in using Java/J2EE tech stacks
• Good understanding of data structures and algorithms.
• Excellent analytical and problem-solving skills.
• Ability to work in a fast-paced internet start-up environment.
• Experience in technical mentorship/coaching is highly desirable.
• Understanding AI/ML algorithms is a plus.

Required Skills:

• 3+ years of experience in the Development of JAVA technology.
• Strong Java Basics
• SpringBoot or Spring MVC
• Hands-on experience in Relational Databases (SQL query or Hibernate) + Mongo (JSON parsing)
• Proficient in REST API development
• Messaging Queue (RabitMQ or Kafka)
• Microservices
• Any Caching Mechanism
• Good at problem-solving

Good to Have Skills:

• 4+ years of experience in using Java/J2EE tech stacks
• Good understanding of data structures and algorithms.
• Excellent analytical and problem-solving skills.
• Ability to work in a fast-paced internet start-up environment.
• Experience in technical mentorship/coaching is highly desirable.
• Understanding AI/ML algorithms is a plus.

Skills Required:

• Must-Have: Java, Spring/spring-boot, Data structure, Algorithm
• Any 2 (Must) out of the below-mentioned list : 

a.     JPA/Hibernate

b.     Messaging queue/Kafka/SQS/distributed message/SNS/JMS

c.     NoSQL/Aerospike/Reddis/Cassandra

d.     Microservices

Roles and Responsibilities:

• Technical design, implementation, deployment, and support.
• Partner with Business Analysts to review and implement business requirements.
• Perform development and unit testing, working closely with Business.
• Mentors and oversees the development of resources, including reviewing designs and performing code reviews.
• Ensure designs are in compliance with specifications
• Contribute to all phases of the development lifecycle
• Developing high-volume, low-latency applications for mission-critical systems and delivering high availability and performance
• Should have experience of working on Core Java/J2EE & OOPS concept
• Should be well versed with Spring/Struts & Apache Camel (OSGI Framework)
• Should have a good understanding of Hibernate and Other ORMs
• Should have an understanding of working on Web Service (SOAP/REST) and Maven
• Build tools such as Jenkins
• Caching Technique(Radis, Hazlecast, Aerospike)
• Database Knowledge - Oracle, MySQL

• Understand long-term goals of AiSensy & design extensive softwares
• Prepare blueprints of backend services from the provided product requirements
• Develop software solutions by studying information needs, conferring with users, and studying systems flow, data usage, and work processes.
• Understand data storage requirements
• Document and demonstrate solutions by developing documentation, flowcharts, layouts, diagrams, charts, code comments and clear code.
• Discover critical challenges in flows, edge cases & perform benchmarking
• Follow the software development lifecycle
• Define success criteria for testing & product acceptance
• Ensures the quality & reliability of deliverables
• Prepare and installs solutions by determining and designing system specifications, standards, and programming.
• Improve operations by conducting systems analysis and recommending changes in policies and procedures.
• Provide information by collecting, analysing, and summarising development and service issues.
• Accomplish engineering and organisation mission by completing related results as needed.
• Experience with non-functional requirements, such as reliability and availability, scale, horizontal scalability, etc. over time and make tech stack decisions accordingly.

• Familiarity with UI/Frontend development using React.js & Redux
• Experience working with service oriented architectures and web-based solutions
• Experience with developing REST APIs & Integrations development using Node.js & MongoDB
• Experience with modelling complex interconnected entities in databases
• Software testing using Jest & Mocha
• Teamwork & Communication skills
• Software development fundamentals
• Experience with working on the AWS technologies - Lambda, SQS, SNS, Eventbridge & API Gateway
• Experience with CI/CD pipelines
• Experience with designing secure distributed systems

• Design, build and maintain efficient, reusable, and reliable Java code. 
• Providing technical oversight to the team, involved in design and code review.
• You would also be spending most of the time in development activities in varied technologies and should have a passion to write code.
• Should be able to create good technical documents.
• Translate application storyboards and use cases into functional applications.
• Ensure the best possible performance, quality, and responsiveness of the applications.
• Identify bottlenecks and bugs, and devise solutions to these problems - Help maintain code quality, organization, and automatization. 
• Prepare the technical design of complex technology components as well as suggest the pros and cons of using a certain technology stack or component or design pattern versus another, to the clients.
• Team Management.

• Qualifications: BTECH/MTECH/MCA/MSc.
• Proficient in Java, with a good knowledge of its ecosystems with a knack for writing clean, readable Java code, writing reusable Java libraries along with knowledge of multithreading, concurrency patterns, and collections in Java. 
• Solid understanding of object-oriented programming along with various design and architectural patterns.
• Hands on experience with JMS, JPA, Spring (MVC, Boot & Cloud preferred) & Hibernate.
• Familiarity with concepts of MVC, JDBC, and RESTful.  Experience with Presentation Layer (JSP/Servlets), JS Frameworks (Angular, jQuery, react, etc.).
• Creating database schemas that represent and support business processes and experience with both external and embedded databases. Implementing automated testing platforms and unit tests.
• Proficient understanding of code versioning tools, such as Git, Ant, Maven, and Gradle; continuous integration.
• Knowledge in xml based mappings, SAML, rest clients, CAS authentication and Jetty.
• Knowledge of Apache Camel and Kafka and Drools Rule Engine is preferred.

Responsibilities:

• Good understanding of programming language
• Concepts of Java, Maven
• Source Control
• Logically Strong in coding practices
• Data structure Understanding.
• Security encryption and decryption
• Data encoding decoding

Requirements:

• Strong core Java development skills.
• Exposure to security tools.
• Experience in working with customers, understanding business requirements
• Good analytical and troubleshooting skills.
• Working knowledge of Jenkins/Maven
• Key requirements: Core Java, CI/CD, Docker, Maven, Jenkins, REST API, Kafka, Redis, cache, elastic cache

• Hands-on expertise in managing technical teams and work closely with business stakeholders
• Should be familiar with card testing and POS ecosystem.
• Experience with designing APIs and building platform integrations in Commerce, POS, FinTech and Payments
• Payment’s domain experience.
• Extensive experience on Base24 & SWITCH part for payment format.
• Exposure on ISO8583 Messaging system.
• API understanding and creation.
• Working knowledge on cryptographic algorithms (RSA, EC, AES, SHA variants), TLS/SSL protocols. C Symetric and A symetrics, K.M.S.
• Digital Payments technical knowledge - QR/ POS/ UPI/ BQR
• Understanding of CI/CD pipelines, Kubernetes, containerization, Docker
• Must have strong technical skills in Android APK and iOS development Concept and Web services.
• Supportability, monitoring, ops in general.
• Accurate cost models to deploy and operate
• Extensive experience in TLOG’s, data integration with downstream applications, and impact of POS changes on different financial, legal, compliance, loss prevention, etc. applications.
• Extensive experience with payment systems 7861 (EMV, payment middleware, encryption, tokenization, settlement, offline scenarios,).
• Extensive experience with different types of payment methods including credit, debit, gift cards (Both closed loop and open loop), PLCC, PayPal, AliPay, checks, etc. as well as different payment form factors like Tap N Go, contactless, swipe, EMV, Mobile, etc.
• Extensive experience with payment devices (pin pads, printers, registers, scanners).
• Experience in Node.js / Java / Python, building REST APIs, with experience/exposure to front end stack like JavaScript & React / Angular. 
• Experience with Storage systems - RDBMS, NoSQL (Cassandra/MongoDB), Redis/Memcache/Aerospike.
• Experience in messaging tools like Kafka/AWS Kinesis etc...
• Design expertise for performance, scalability, fault tolerance and high availability

• Experience with POS systems and Payments domain.
• Demonstrated experience for systems architecture, design, and integration for POS systems
• Java and/or Node.js. Experience building micro services (REST APIs) using Node.js
• Experience with one or more RDBMS databases like MySql, PostGreSQL
• Experience with one or more NoSQL databses like MongoDB, Cassandra, DynamoDB, Aerospike, Elastic Search, Redis, neo4j etc.,
• Caching technologies like: Redis, Elasticache,
• Cloud Providers like: AWS/ Google Cloud / Azure
• CI / CD : Jenkins, Docker, Kubernete
  

We are looking for a bright and exceptional Engineering Lead to join one of our Fortune 500 clients in the Fintech industry for the locations - Bangalore / Gurgaon / Hyderabad based Technology team. The role involves building complex next generation product used by the clients and architecting solutions to support new technical and business initiatives.

Responsibilities:

• Active participation in design, development, delivery, and maintenance of software projects/products.
• Help the team translate the business requirements into R& D tasks.
• Work with product groups to outline project deliverables and help manage the roadmap of the R& D tasks.
• Act as a point of contact for managing and driving production defects & stability improvements to resolution.
• Tailor processes to help manage time-sensitive issues and bring them to appropriate closure.
• Engage and lead a team of highly talented technologists.

Requirements:

• A bachelor's degree in Computer Science with 6-9 years of experience, Fintech domain is a plus.
• Demonstrated contribution to end-to-end delivery of enterprise-grade software.
• Proficiency in Java or equivalent object-oriented languages, coupled with design and SOA.
• Hands-on experience in enterprise application development, coupled with knowledge of frameworks & tools like/equivalent to spring, mybatis, git, gradle
• Experience in at least one shell scripting language, SQL, and data modeling skills.
• Experience in cloud technologies is a plus.
• Strong technology acumen, knowledge of software engineering process, design knowledge and architecture intelligence.
• Attention to detail and quality, and the ability to work well in and across teams.
• Ability to advocate & influence multiple stakeholders.
• Excellent analytical and reasoning skills.
• Ability to learn new domains and deliver output.
• Experience in leading a team of highly skilled engineers.
• Strong communication skills.
• The ideal candidate should have strong problem solving and analytical skills as well as a demonstrated passion for technology.
• Excellent reasoning ability and good interpersonal and communication skills are essential for this role as it involves interaction with business users.

Developer (SQL & Python) -
Required Technical Skills

• At least 5+ years’ experience as a software engineer
• Experience in Python is a must
• Experience working with relational/non-relational databases and understanding of storage technologies (like MySQL, Sybase, MongoDB, InfluxDB, Cassandra or HBase)
• Experience / Familiarity with Database Modelling, Normalization techniques
• Experience / Familiarity with object-oriented design patterns
• Ability to trouble shoot and fix performance issues across the codebase and database queries

Preferred Skills

• BA/BS in Computer Science or equivalent practical experience
• Experience with dev ops tools like Git, Maven, Jenkins, Gitlab CI, Azure DevOps
• Experience with Agile development concepts and related tools
• Ability to trouble shoot and fix performance issues across the codebase and database queries
• Experience with Python frameworks utilizing Asyncio
• Experience working with large volumes of time series data and building services, APIs and applications based of it
• Experience in designing multi-tier application architecture and distributed caching solutions
• Experience with Perl
• ETL background in any language or tools
• Experience with cloud technologies like Kubernetes, Docker, OpenStack and Kafka
• Experience with web technologies like Angular 2+ (or React/Vue), TypeScript, RxJS
• Experience with Go

Roles & Responsibilities

• Designing and delivering a best-in-class, highly scalable data governance platform
• Improving processes and applying best practices
• Contribute in all scrum ceremonies; assuming the role of ‘scum master’ on a rotational basis
•  Development, management and operation of our infrastructure to ensure it is easy to deploy, scalable, secure and fault-tolerant
• Flexible on working hours as per business needs

• Exp- 3+ yrs
• Must be very good in Java- 2.5+ yrs
• At least 1 yr of working experience in any one DB of Neo4j, Cassandra, Elastic Search
• Should have good devops working knowledge - Knowledge of AWS, Ansibleetc is a necessity
• Experience in TDD/BDD is required
• Minimum 1 yr working experience in Samza& Kafka.
• Knowledge of Azure is added advantage
• Understanding of AKKA and Play framework

Who we are

ixigo is an AI-based travel app that helps millions of users plan and book their trips every day. Launched in 2007, ixigo is proud to have built numerous industry-first AI-innovations over the last decade. With pre-emptive help from ixigo at every touch-point of their journey, 150 million Indians are now making smarter travel decisions every year. ixigo uses Artificial Intelligence for deal discovery, personalized recommendations, dynamic marketing, airfare predictions, train delay information, PNR confirmation status, infrastructure cost optimization, business intelligence and providing fully-automated customer service over chat and voice. This makes ixigo a lean, tech-intensive and capital-efficient travel business, serving millions of travelers with a team size of under 200. Founded in 2007 by Aloke Bajpai & Rajnish Kumar, ixigo’s investors include Sequoia Capital India, Fosun RZ Capital, SAIF Partners, MakeMyTrip & Micromax.

Role & Responsibility

As a Backend Developer, You will be involved in all aspects of development playing a critical role in the design, planning, development, and deployment and help develop best practices for future development. You will be working closely with Tech, Design and Quality team to build and launch the product.

• Code, develop, design and architect features related to the existing application. We expect you to own modules end to end and take pride in the work products you ship
• Design, implement and maintain highly scalable, multi-tier, distributed server software applications
• Deliver across the entire app life cycle, design, build, deploy, test, release to app stores and support
• Working directly with developers and product managers to conceptualize, build, test and release products
• Take responsibility for performance and stability for all server functions
• Continuously discover, evaluate, and implement new technologies to maximize development efficiency

Skills & Qualifications:

• At least an undergraduate degree in Computer Science, Engineering, preferably from IITs/Top RECs
• 2-5 years of experience at cutting edge organizations or startups
• Extensive experience in a Java server-side software design and development
• Proficiency in the implementation of algorithms and the use of advanced data structures to solve problems in computing.
• Experience with e-business/e-commerce applications and business environment is a must
• Any experience with internet companies or the travel industry will be a plus

Must have

• Good understanding of Java/J2EE and design patterns
• Strong knowledge of data structures and designing for performance, scalability and availability Knowledge in MongoDB and Web services
• Extensive use of Instruments and Shark to detect memory leaks and for performance optimization
• Good grasp of algorithms, memory management, and multithreaded programming
• Good to have - Python, NodeJS, Mysql, Redis, ElasticSearchGreat people skills!
• Able to fit in well within an informal startup environment and to provide hands-on management
• The high energy level and untiring commitment to drive oneself & the team towards goals

Compensation: Commensurate to skills & experience; comparable to the best in the industry.

Culture: ixigo is proud to have built an entrepreneurial culture that has become a folk-lore in the startup ecosystem. One in every four ixigems has gone on to build successful startups and companies. Our cultural values of integrity, empathy, ingenuity, awesomeness, and resilience have stood the tests of time and we’ve built a fun, flexible and creative work environment that is driven by people with a high degree of ownership. You will get to work with some of the smartest folks in the Indian startup ecosystem, and solve some of the toughest problems for the next billion users by using bleeding-edge technologies. Oh, and we have an awesome “play” area, great chai/coffee, free lunches (yes, they exist!) and a workspace you will fall in love with.