Apache Kafka Jobs in Mumbai

About the Role We are looking for a Data Engineer to help us scale the existing data infrastructure and in parallel work on building the next generation data platform for analytics at scale, machine learning infrastructure and data validation systems.In this role, you will be responsible for communicating effectively with data consumers to fine-tune data platform systems (existing or new), taking ownership and delivering high performing systems and data pipelines, and helping the team scale them up, to endure ever growing traffic.This is a growing team, which makes for many opportunities to be involved directly with product management, development, sales, and support teams. Everybody on the team is passionate about their work and we’re looking for similarly motivated “get stuff done” kind of people to join us! Roles & Responsibilities Engineer data pipelines (batch and real-time ) that aids in creation of data-driven products for our platform Design, develop and maintain a robust and scalable data-warehouse and data lake Work closely alongside Product managers and data-scientists to bring the various datasets together and cater to our business intelligence and analytics use-cases Design and develop solutions using data science techniques ranging from statistics, algorithms to machine learning Perform hands-on devops work to keep the Data platform secure and reliable Skills Required Bachelor's degree in Computer Science, Information Systems, or related engineering discipline 6 + years’ experience with ETL, Data Mining, Data Modeling, and working with large-scale datasets 6+ years’ experience with an object-oriented programming language such as Python, Scala, Java, etc Extremely proficient in writing performant SQL working with large data volumes Experience with map-reduce, Spark, Kafka, Presto, and the ecosystem.  Experience in building automated analytical systems utilizing large data sets. Experience with designing, scaling and optimizing cloud based data warehouses (like AWS Redshift) and data lakes Familiarity with AWS technologies preferred Qualification – B.Tech/M.Tech/MCA(IT/Computer Science) Years of Exp – 6-9

About Us upGrad is an online education platform building the careers of tomorrow by offering the most industry-relevant programs in an immersive learning experience. Our mission is to create a new digital-first learning experience to deliver tangible career impact to individuals at scale. upGrad currently offers programs in Data Science, Machine Learning, Product Management, Digital Marketing, and Entrepreneurship, etc.  upGrad is looking for people passionate about management and education to help design learning programs for working professionals to stay sharp and stay relevant and help build the careers of tomorrow. upGrad was awarded the Best Tech for Education by IAMAI for 2018-19 upGrad was also ranked as one of the LinkedIn Top Startups 2018: The 25 most sought-after startups in India  upGrad was earlier selected as one of the top ten most innovative companies in India by FastCompany. We were also covered by the Financial Times along with other disruptors in Ed-Tech upGrad is the official education partner for Government of India - Startup India program  Our program with IIIT B has been ranked #1 program in the country in the domain of Artificial Intelligence and Machine Learning    About the Role A highly motivated and passionate individual who has experience in executing end to end web based application projects and bringing them to production quickly and with high quality. Passionate about building flexible and scalable solutions with an eye for detail and can weigh pros and cons to find best possible solutions.  Role and Responsibilities Collaborate with Product Manager and Engineering Manager to plan, implement and deliver tasks and modules on a timely basis with best practices and adherence to SOPs. Understand the product specification and be able to ask questions and gather information and feedback to build a well rounded solution.  Deliver end to end applications starting from front end web development, back end development, middleware integration and persisting data in no SQL or RDBMS databases. Should be able to use existing front-end components to build web based UI interfaces. Strong technical design approach to problem solving for backend services. Should be able to communicate and defend the tech design choices taken for long term scalability and maintainability. Play the role of an unblocker in a tight scrum environment. Should be able to help other developers with challenges and help achieve milestones as per plan. Prioritize to manage ad-hoc requests in parallel with ongoing projects. Document API specification, technical design and other key information for all projects/modules/services owned. Skills/Experience A highly talented developer with 8+ years of hands-on experience in building secure and scalable web based applications using Java in backend and JS based frontend frameworks. Strong knowledge of Java 8+, databases (SQL and NoSQL) , MySql, Mongo,  Experience working with Cassandra, Spring Boot, Apache Spark, Kafka, RabbitMQ is highly desirable Working knowledge of any one or more of AngularJS/ReactJS/VueJS and CSS3, HTML5. Strong experience in data modelling for data storage tools in SQL; experience with noSQL is highly desirable.  Experience building scalable solutions for 2x, 5x and 10x growth with high quality and flexibility. Practitioner of test driven development practices. Experience in creating and adhering to best practices for development. Experience working with container technologies like Docker and knowledge of CI/CD processes for automated integration and deployment. Basic knowledge of AWS preferred. Strong experience in Microservices (SRP) paradigm. Willing to learn new technologies (Golang) Qualification – B.Tech/M.Tech/MCA(IT/Computer Science) Years of Exp – 8-10+ years

Minimum experience of 4+ years working on Java / Scala Minimum 4 years of experience in working on Hadoop / Spark is a must. Sound knowledge of building restful APIs Hands on experience on rdbms and nosql databases Hands on experience on AWS / GCP infrastructure such as EMR, Athena, RedShift, DynamoDb etc. Good understanding of data security Should have executed a minimum of one project in recommendation engine for retail / media / insurance or any other domain Hands on experience on streaming platforms like Kafka / Flink / Storm should be able to build infrastructure for ETL and feature extractions Knowledge of machine learning techniques like classification, clustering, decision tree, NLP is desirable Working knowledge of python / R is bonus

Position – Engineering Manager Location – Bangalore/Mumbai/Remote   About Us upGrad is an online education platform building the careers of tomorrow by offering the most industry-relevant programs in an immersive learning experience. Our mission is to create a new digital-first learning experience to deliver tangible career impact to individuals at scale. upGrad currently offers programs in Data Science, Machine Learning, Product Management, Digital Marketing, and Entrepreneurship, etc.   upGrad is looking for people passionate about management and education to help design learning programs for working professionals to stay sharp and stay relevant and help build the careers of tomorrow.   upGrad was awarded the Best Tech for Education by IAMAI for 2018-19 upGrad was also ranked as one of the LinkedIn Top Startups 2018: The 25 most sought-after startups in India upGrad was earlier selected as one of the top ten most innovativecompanies in India by FastCompany. We were also coveredby the Financial Times along with other disruptors in Ed-Tech upGrad is the official education partnerfor Government of India - Startup India program  Our program with IIIT B has been ranked #1 programin the country in the domain of Artificial Intelligence and Machine Learning    About the Role A highly motivated and passionate individual who has experience in executing end to end web based products and applications; bringing them to production quickly and with high quality. Passionate about building flexible and scalable solutions with an eye for detail and can weigh pros and cons to find best possible solutions.   Role and Responsibilities   Collaborate with Product Managers to plan, implement and deliver tasks and modules on a timely basis with best practices and adherence to SOPs. Understand the product requirements, ask questions, and gather information and feedback to design and deliver scalable and flexible systems. Deliver end to end applications from front end web development, back end development, middleware integration and persisting data in no SQL or RDBMS databases. Play the role of an unblocker in a tight scrum environment. Should be able to help other developers with challenges, problem solving, and help achieve milestones as per plan. Build a highly skilled and efficient team by hiring, keeping the team motivated, and managing performance. Build a culture of agile development, continuous improvement, and a strong sense of product ownership Drive a culture of high quality at all stages by leveraging CI/CD and automation. Identify architectural issues and plans to resolve them. Proactively identify risks and define mitigations plans. Prioritize to manage ad-hoc requests in parallel with ongoing projects. Hands on with programming with 25%+ of bandwidth going into coding activities owned.     Skills/Experience A highly talented developer with 10+ years of hands-on experience in building secure and scalable web based applications using Java in backend and JS based frontend frameworks. Demonstrated experience managing teams of 5-10 or more engineers in product or online teams. Strong knowledge of Java 8+, databases (SQL and NoSQL) , MySql, Mongo, Experience working with Cassandra, Spring Boot, Apache Spark, Kafka, RabbitMQ is highly desirable Working knowledge of any one or more of AngularJS/ReactJS/VueJS and CSS3, HTML5. Strong experience in data modelling for data storage tools in SQL; experience with noSQL is highly desirable. Experience building scalable solutions for 2x, 5x and 10x growth with high quality and flexibility. Practitioner of test driven development practices. Experience in creating and adhering to best practices for development. Experience working with container technologies like Docker and knowledge of CI/CD processes for automated integration and deployment. Basic knowledge of AWS preferred. Strong experience in Microservices (SRP) paradigm. Willing to learn new technologies (Golang)       Qualification – B.Tech/M.Tech/MCA(IT/Computer Science) Years of Exp – 10+

JOB DESCRIPTION: We are looking for a Data Engineer with a solid background in scalable systems to work with our engineering team to improve and optimize our platform. You will have significant input into the team’s architectural approach and execution. We are looking for a hands-on programmer who enjoys designing and optimizing data pipelines for large-scale data. This is NOT a "data scientist" role, so please don't apply if you're looking for that. RESPONSIBILITIES: 1. Build, maintain and test, performant, scalable data pipelines 2. Work with data scientists and application developers to implement scalable pipelines for data ingest, processing, machine learning and visualization 3. Building interfaces for ingest across various data stores MUST-HAVE: 1. A track record of building and deploying data pipelines as a part of work or side projects 2. Ability to work with RDBMS, MySQL or Postgres 3. Ability to deploy over cloud infrastructure, at least AWS 4. Demonstrated ability and hunger to learn GOOD-TO-HAVE: 1. Computer Science degree 2. Expertise in at least one of: Python, Java, Scala 3. Expertise and experience in deploying solutions based on Spark and Kafka 4. Knowledge of container systems like Docker or Kubernetes 5. Experience with NoSQL / graph databases: 6. Knowledge of Machine Learning Kindly apply only if you are skilled in building data pipelines.