20+ Hadoop Jobs in Mumbai | Hadoop Job openings in Mumbai

We are seeking a skilled and innovative Developer with strong expertise in Scala, Java/Python and Spark/Hadoop to join our dynamic team.

Key Responsibilities:

• Design, develop, and maintain robust and scalable backend systems using Scala, Spark, Hadoop and expertise in Python/Java.

• Build and deploy highly efficient, modular, and maintainable microservices architecture for enterprise-level applications.

• Write and optimize algorithms to enhance application performance and scalability.

Required Skills:

• Programming: Expert in Scala and object-oriented programming.

• Frameworks: Hands-on experience with Spark and Hadoop

• Databases: Experience with relational databases (e.g., PostgreSQL, MySQL) and NoSQL databases (e.g., MongoDB).

• Location: Mumbai

• Employment Type: Full-time

Work Mode: Hybrid

Need B.Tech, BE, M.Tech, ME candidates - Mandatory

Must-Have Skills:

● Educational Qualification :- B.Tech, BE, M.Tech, ME in any field.

● Minimum of 3 years of proven experience as a Data Engineer.

● Strong proficiency in Python programming language and SQL.

● Experience in DataBricks and setting up and managing data pipelines, data warehouses/lakes.

● Good comprehension and critical thinking skills.

● Kindly note Salary bracket will vary according to the exp. of the candidate - 

- Experience from 4 yrs to 6 yrs - Salary upto 22 LPA

- Experience from 5 yrs to 8 yrs - Salary upto 30 LPA

- Experience more than 8 yrs - Salary upto 40 LPA

About the company

DCB Bank is a new generation private sector bank with 442 branches across India.It is a scheduled commercial bank regulated by the Reserve Bank of India. DCB Bank’s business segments are Retail banking, Micro SME, SME, mid-Corporate, Agriculture, Government, Public Sector, Indian Banks, Co-operative Banks and Non-Banking Finance Companies.

Job Description

Department: Risk Analytics

CTC: Max 18 Lacs

Grade: Sr Manager/AVP

Experience: Min 4 years of relevant experience

We are looking for a Data Scientist to join our growing team of Data Science experts and manage the processes and people responsible for accurate data collection, processing, modelling, analysis, implementation, and maintenance.

Responsibilities

• Understand, monitor and maintain existing financial scorecards (ML Based) and make changes to the model when required.
• Perform Statistical analysis in R and assist IT team with deployment of ML model and analytical frameworks in Python.
• Should be able to handle multiple tasks and must know how to prioritize the work.
• Lead cross-functional projects using advanced data modelling and analysis techniques to discover insights that will guide strategic decisions and uncover optimization opportunities.
• Develop clear, concise and actionable solutions and recommendations for client’s business needs and actively explore client’s business and formulate solutions/ideas which can help client in terms of efficient cost cutting or in achieving growth/revenue/profitability targets faster.
• Build, develop and maintain data models, reporting systems, data automation systems, dashboards and performance metrics support that support key business decisions.
• Design and build technical processes to address business issues.
• Oversee the design and delivery of reports and insights that analyse business functions and key operations and performance metrics.
• Manage and optimize processes for data intake, validation, mining, and engineering as well as modelling, visualization, and communication deliverables.
• Communicate results and business impacts of insight initiatives to the Management of the company.

Requirements

• Industry knowledge
• 4 years or more of experience in financial services industry particularly retail credit industry is a must.
• Candidate should have either worked in banking sector (banks/ HFC/ NBFC) or consulting organizations serving these clients.
• Experience in credit risk model building such as application scorecards, behaviour scorecards, and/ or collection scorecards.
• Experience in portfolio monitoring, model monitoring, model calibration
• Knowledge of ECL/ Basel preferred.
• Educational qualification: Advanced degree in finance, mathematics, econometrics, or engineering.
• Technical knowledge: Strong data handling skills in databases such as SQL and Hadoop. Knowledge with data visualization tools, such as SAS VI/Tableau/PowerBI is preferred.
• Expertise in either R or Python; SAS knowledge will be plus.

Soft skills:

• Ability to quickly adapt to the analytical tools and development approaches used within DCB Bank
• Ability to multi-task good communication and team working skills.
• Ability to manage day-to-day written and verbal communication with relevant stakeholders.
• Ability to think strategically and make changes to data when required.

Key Responsibilities:

• Install, configure, and maintain Hadoop clusters.

• Monitor cluster performance and ensure high availability.

• Manage Hadoop ecosystem components (HDFS, YARN, Ozone, Spark, Kudu, Hive).

• Perform routine cluster maintenance and troubleshooting.

• Implement and manage security and data governance.

• Monitor systems health and optimize performance.

• Collaborate with cross-functional teams to support big data applications.

• Perform Linux administration tasks and manage system configurations.

• Ensure data integrity and backup procedures.

Role: Principal Software Engineer

We looking for a passionate Principle Engineer - Analytics to build data products that extract valuable business insights for efficiency and customer experience. This role will require managing, processing and analyzing large amounts of raw information and in scalable databases. This will also involve developing unique data structures and writing algorithms for the entirely new set of products. The candidate will be required to have critical thinking and problem-solving skills. The candidates must be experienced with software development with advanced algorithms and must be able to handle large volume of data. Exposure with statistics and machine learning algorithms is a big plus. The candidate should have some exposure to cloud environment, continuous integration and agile scrum processes.

Responsibilities:

• Lead projects both as a principal investigator and project manager, responsible for meeting project requirements on schedule

• Software Development that creates data driven intelligence in the products which deals with Big Data backends

• Exploratory analysis of the data to be able to come up with efficient data structures and algorithms for given requirements

• The system may or may not involve machine learning models and pipelines but will require advanced algorithm development

• Managing, data in large scale data stores (such as NoSQL DBs, time series DBs, Geospatial DBs etc.)

• Creating metrics and evaluation of algorithm for better accuracy and recall

• Ensuring efficient access and usage of data through the means of indexing, clustering etc.

• Collaborate with engineering and product development teams.

Requirements:

• Master’s or Bachelor’s degree in Engineering in one of these domains - Computer Science, Information Technology, Information Systems, or related field from top-tier school

• OR Master’s degree or higher in Statistics, Mathematics, with hands on background in software development.

• Experience of 8 to 10 year with product development, having done algorithmic work

• 5+ years of experience working with large data sets or do large scale quantitative analysis

• Understanding of SaaS based products and services.

• Strong algorithmic problem-solving skills

• Able to mentor and manage team and take responsibilities of team deadline.

Skill set required:

• In depth Knowledge Python programming languages

• Understanding of software architecture and software design

• Must have fully managed a project with a team

• Having worked with Agile project management practices

• Experience with data processing analytics and visualization tools in Python (such as pandas, matplotlib, Scipy, etc.)

• Strong understanding of SQL and querying to NoSQL database (eg. Mongo, Casandra, Redis

Role / Purpose - Lead Developer - API and Microservices

Must have a strong hands-on development track record building integration utilizing a variety of integration products, tools, protocols, technologies, and patterns.

• Must have an in-depth understanding of SOA/EAI/ESB concepts, SOA Governance, Event-Driven Architecture, message-based architectures, file sharing, and exchange platforms, data virtualization and caching strategies, J2EE design patterns, frameworks
• Should possess experience with at least one of middleware technologies (Application Servers, BPMS, BRMS, ESB & Message Brokers), Programming languages (e.g. Java/J2EE, JavaScript, COBOL, C), Operating Systems (e.g. Windows, Linux, MVS), and Databases (DB2, MySQL, No SQL Databases like MongoDB, Cassandra, Hadoop, etc.)
• Must have experience implementing API Service architectures (SOAP, REST) using any of the market-leading API Management tools such as Apigee and frameworks such as Spring Boot for Microservices
• Should have Advanced skills in implementing API Service architectures (SOAP, REST) using any of the market-leading API Management tools such as Apigee or similar frameworks such as Spring Boot for Microservices 
• Appetite to manage large-scale projects and multiple tracks
•  Experience and knowhow of the e-commerce domain and retail experience are preferred
•  Good communication & people managerial skills

Job Description for :

Role: Data/Integration Architect

Experience – 8-10 Years

Notice Period: Under 30 days

Key Responsibilities: Designing, Developing frameworks for batch and real time jobs on Talend. Leading migration of these jobs from Mulesoft to Talend, maintaining best practices for the team, conducting code reviews and demos.

Core Skillsets:

Talend Data Fabric - Application, API Integration, Data Integration. Knowledge on Talend Management Cloud, deployment and scheduling of jobs using TMC or Autosys.

Programming Languages - Python/Java
Databases: SQL Server, Other Databases, Hadoop

Should have worked on Agile

Sound communication skills

Should be open to learning new technologies based on business needs on the job

Additional Skills:

Awareness of other data/integration platforms like Mulesoft, Camel

Awareness Hadoop, Snowflake, S3

• upGrad was awarded the Best Tech for Education by IAMAI for 2018-19,
• upGrad was also ranked as one of the LinkedIn Top Startups 2018: The 25 most sought-after startups in India.
• upGrad was earlier selected as one of the top ten most innovative companies in India by FastCompany.
• We were also covered by the Financial Times along with other disruptors in Ed-Tech.
• upGrad is the official education partner for Government of India - Startup India program.
• Our program with IIIT B has been ranked #1 program in the country in the domain of Artificial Intelligence and Machine Learning.

• Work with Product Managers and Business to understand business/product requirements and vision.
• Provide a clear architectural vision in line with business and product vision.
• Lead a team of architects, developers, and data engineers to provide platform services to other engineering teams.
• Provide architectural oversight to engineering teams across the organization.
• Hands on design and development of platform services and features owned by self - this is a hands-on coding role.
• Define guidelines for best practices covering design, unit testing, secure coding etc.
• Ensure quality by reviewing design, code, test plans, load test plans etc. as appropriate.
• Work closely with the QA and Support teams to track quality and proactively identify improvement opportunities.
• Work closely with DevOps and IT to ensure highly secure and cost optimized operations in the cloud.
• Grow technical skills in the team - identify skill gaps with plans to address them, participate in hiring, mentor other architects and engineers.
• Support other engineers in resolving complex technical issues as a go-to person.

• 12+ years of experience in design and development of ecommerce scale systems and highly scalable SaaS or enterprise products.
• Extensive experience in developing extensible and scalable web applications with
  • Java, Spring Boot, Go
  • Web Services - REST, OAuth, OData
  • Database/Caching - MySQL, Cassandra, MongoDB, Memcached/Redis
  • Queue/Broker services - RabbitMQ/Kafka
  • Microservices architecture via Docker on AWS or Azure.
  • Experience with web front end technologies - HTML5, CSS3, JavaScript libraries and frameworks such as jQuery, AngularJS, React, Vue.js, Bootstrap etc.

• Extensive experience with cloud based architectures and how to optimize design for cost.
• Expert level understanding of secure application design practices and a working understanding of cloud infrastructure security.
• Experience with CI/CD processes and design for testability.
• Experience working with big data technologies such as Spark/Storm/Hadoop/Data Lake Architectures is a big plus.
• Action and result-oriented problem-solver who works well both independently and as part of a team; able to foster and develop others' ideas as well as his/her own.
• Ability to organize, prioritize and schedule a high workload and multiple parallel projects efficiently.
• Excellent verbal and written communication with stakeholders in a matrixed environment.
• Long term experience with at least one product from inception to completion and evolution of the product over multiple years.

At Karza technologies, we take pride in building one of the most comprehensive digital onboarding & due-diligence platforms by profiling millions of entities and trillions of associations amongst them using data collated from more than 700 publicly available government sources. Primarily in the B2B Fintech Enterprise space, we are headquartered in Mumbai in Lower Parel with 100+ strong workforce. We are truly furthering the cause of Digital India by providing the entire BFSI ecosystem with tech products and services that aid onboarding customers, automating processes and mitigating risks seamlessly, in real-time and at fraction of the current cost.

A few recognitions:

• Recognized as Top25 startups in India to work with 2019 by LinkedIn
• Winner of HDFC Bank's Digital Innovation Summit 2020
• Super Winners (Won every category) at Tecnoviti 2020 by Banking Frontiers
• Winner of Amazon AI Award 2019 for Fintech
• Winner of FinTech Spot Pitches at Fintegrate Zone 2018 held at BSE
• Winner of FinShare 2018 challenge held by ShareKhan
• Only startup in Yes Bank Global Fintech Accelerator to win the account during the Cohort
• 2nd place Citi India FinTech Challenge 2018 by Citibank
• Top 3 in Viacom18's Startup Engagement Programme VStEP

What your average day would look like:

• Deploy and maintain mission-critical information extraction, analysis, and management systems
• Manage low cost, scalable streaming data pipelines
• Provide direct and responsive support for urgent production issues
• Contribute ideas towards secure and reliable Cloud architecture
• Use open source technologies and tools to accomplish specific use cases encountered within the project
• Use coding languages or scripting methodologies to solve automation problems
• Collaborate with others on the project to brainstorm about the best way to tackle a complex infrastructure, security, or deployment problem
• Identify processes and practices to streamline development & deployment to minimize downtime and maximize turnaround time

What you need to work with us:

• Proficiency in at least one of the general-purpose programming languages like Python, Java, etc.
• Experience in managing the IAAS and PAAS components on popular public Cloud Service Providers like AWS, Azure, GCP etc.
• Proficiency in Unix Operating systems and comfortable with Networking concepts
• Experience with developing/deploying a scalable system
• Experience with the Distributed Database & Message Queues (like Cassandra, ElasticSearch, MongoDB, Kafka, etc.)
• Experience in managing Hadoop clusters
• Understanding of containers and have managed them in production using container orchestration services.
• Solid understanding of data structures and algorithms.
• Applied exposure to continuous delivery pipelines (CI/CD).
• Keen interest and proven track record in automation and cost optimization.

Experience:

• 1-4 years of relevant experience
• BE in Computer Science / Information Technology